Beiträge von leee

  • REST API: Keycloak "ID" can be trivially confused with Keycloak "preferred_username" which is netcup "Customer number"

    Hi [netcup] Sarah G. - thank you for organizing the issues and reaching out to the relevant departments `:D` Looking forward to updates for them.

    Regarding this issue: for clarification, how do CCP and SCP interact with each other here? Is the following guess correct?

    • netcup "customer number" is from CCP,
    • but SCP has a different set of users with its own SCP Keycloak user IDs
    • so CCP "customer number" is used as SCP Keycloak "username" and we log into SCP control panel and SCP Keycloak using CCP customer number

    Would it be possible to also document this somewhere, such as in the authentication part of the REST API documentation?

    An additional question for clarity: are ID numbers in SCP REST API guaranteed to be unique for the duration of their life? In other words:

    Code
    $ netcup auth print-access-token | jq -R 'split(".") | .[1] | @base64d | fromjson | .id'
"15...." (SCP user ID)

    and

    Code
    $ netcup servers list
ID       NAME                   HOSTNAME               NICKNAME               DISABLED   TEMPLATE
728538   v2202505273910341302   hostname.example.com   nickname.example.com   false      1041 (VPS 1000 ARM G11)
728540   v2202505273910341303   hostname.example.net   nickname.example.net   false      1041 (VPS 1000 ARM G11)

    Are those ID numbers:

    • 15....: User id
    • 728538, 728540: Server id
    • and other IDs including but not limited to: FailoverIPv4 id, FailoverIPv6 id, SSHKey id, VLan id, S3 ids/keys, etc.

    guaranteed to be the same and never change? Or should users of the REST APi always use `GET` methods to list items, find what they want, and reference the ID provided then? What would be the lifetime of that ID if it is considered temporary?

    I also just want to mention these unanswered issues, but I assume you did not get a chance to review and notify the relevant department(s) yet:

    Thank you, and have a great weekend! 🦆

  • REST API: various null fields violate netcup-defined OpenAPI spec

    Hi [netcup] Sarah G. - tonight I was going to work on implementing `netcup servers get ...` and `netcup servers set ...` but found another similar nullable OpenAPI spec violation:

    Code
    $ curl -s -H "Authorization: Bearer $(go run cmd/netcup/main.go auth print-access-token)" \
  "https://www.servercontrolpanel.de/scp-core/api/v1/servers/......" | jq '.serverLiveInfo.interfaces[0].vlanId'
null

    This of course again is not deemed possible in your spec. All your endpoints should be reviewed and either the spec fixed or server behavior fixed.

    Do you already have engineers assigned to work on the various REST API bug fixes? Would it be possible to get a soft timeline on the remediation efforts?

    I will pause on my netcup CLI tool and OpenTofu (Terraform) provider work until we hear back on the state and timeline of further REST API development, and look forward to resuming when the API is actually ready for production use.

    Thank you! 🦆 (I have seen the duck emoji used in many places in the forum, what does it stand for? :) )

  • REST API: IPv6 rDNS endpoints are not well implemented and need a rewrite

    Zitat von crunch

    i suggest to send this to the support guys directly. this is a customer forum.

    I understand this forum section is for feedback for the REST API which only recently exited beta (and possibly a little too early), and I assume [netcup] Sarah G. or another employee will triage issues into their infrastructure or developer experience team projects. If that is not the case, do let me know what the best venue for escalating structural issues, if it is something other than support tickets.

  • REST API: IPv6 rDNS endpoints are not well implemented and need a rewrite

    The IPv6 rDNS endpoints act strangely and frankly need a v2 rewrite.

    Issue 1: addresses are treated as string literals and not normalized

    As I understand things based on testing:

    • IPv6 addresses are treated as string literals in storage/database
    • The addresses are validated (with regex according to the error message)
    • But they are not normalized to some canonical RFC 5952 form

    Example:

    These are the same addresses. Their most compact and expanded forms are:

    Code
    2a0a:4cc0:2000:8fe5:cafe:: / 2a0a:4cc0:2000:8fe5:cafe:0000:0000:0000
2a0a:4cc0:2000:8fe5:f00d:: / 2a0a:4cc0:2000:8fe5:f00d:0000:0000:0000

    The expectation then is we should see cafe be test-4.example.com, and f00d be test-0.example.net. And sure enough, after waiting for the clock to go past a multiple of 10 minutes (and a few seconds), we check what is actually being served from DNS:

    Code
    $ dig @root-dns.netcup.net +noall +answer -x 2a0a:4cc0:2000:8fe5:cafe::
0.0.0.0.0.0.0.0.0.0.0.0.e.f.a.c.5.e.f.8.0.0.0.2.0.c.c.4.a.0.a.2.ip6.arpa. 86400    IN PTR test-4.example.com.

$ dig @root-dns.netcup.net +noall +answer -x 2a0a:4cc0:2000:8fe5:f00d::
0.0.0.0.0.0.0.0.0.0.0.0.d.0.0.f.5.e.f.8.0.0.0.2.0.c.c.4.a.0.a.2.ip6.arpa. 86400    IN PTR test-0.example.net.

    As you can see, the responses show that the most recent POST is what gets committed to DNS configuration. This is perfectly fine and expected.

    What's not acceptable is this API response:

    Like we state before, all these addresses are the same. While it is possible (but typically recommended) to have multiple PTR records for the same address, that is not what is going on here. There should only be 1 instance of each address, even if in the future they can be for multiple PTR records. It doesn't matter if you store records compact or expanded, that mechanism needs to be consistent to avoid ambiguities like the above.

    Issue 2: IPv6 GET endpoint is not for a singular address, but for a subnet prefix (with an undisclosed /64 prefix mask)

    Related to the above issue, when I `rdns get 2a0a:4cc0:2000:8fe5::` I expect to get the PTR record(s) for the address `2a0a:4cc0:2000:8fe5:0000:0000:0000:0000`. Instead, what we get is a response that we would've expected from a command like `rdns list` or `rdns list 2a0a:4cc0:2000:8fe5::/64`.

    At minimum, this behavior should be documented. However, I believe these endpoints should be written to be consistent, so like the IPv4 GET /api/v1/rdns/ipv4/{ip} endpoint, the IPv6 GET /api/v1/rdns/ipv6/{ip} endpoint is for a single address.

    Issue 3: IPv4 and IPv6 endpoints are not consistent with each other

    Similarly to the above issue:

    • IPv4 `GET` returns just the PTR record, IPv6 `GET` returns both address and PTR record (see related next issue)
    • /api/v1/rdns/ipv4/{ip} and /api/v1/rdns/ipv6/{ip} should have the `POST` method since they already also have the `GET` and `DELETE`
    • /api/v1/rdns/ipv4 and /api/v1/rdns/ipv6 can then optionally be repurposed for retrieving all rDNS records associated with the user, similar to /api/v1/servers (but it can be argued that this is also ambiguous, and should be remade as /api/v1/users/{userId}/servers instead, unless there is a plan for org-based IAM in the future)

    Issue 4: deleting an IPv6 PTR record sets it to `nobody.yourvserver.net` and not back to nothing

    Similar to what I mentioned in another post regarding IPv4.

    IPv6 addresses start out without any PTR records. Then the user adds one. The user then deletes that record, causing the address to have a `nobody.yourvserver.net` PTR record just like IPv4.

    This is inconsistent behavior. Addresses that no longer have any PTR records in the REST API should stay that way.

  • REST API: PTR record does not align with REST API state for deleted rdns (or even SCP)

    After using `DELETE /api/v1/rdns/ipv4/{ip}` on a vServer IP with the REST API, I would expect for there to be no PTR record. However:

    Code
    $ # wait 10 minutes for netcup infrastructure to deploy changed records
$ dig @root-dns.netcup.net +short -x 198.51.100.100
nobody.yourvserver.net.

    If netcup would like `nobody.yourvserver.net` to be a default PTR record for servers that have no explicitly defined hostname, the REST API should also report this on `GET`.

    It is also especially weird that SCP does not have the updated hostname either. Is that also on a batched job like the every 10 minute DNS server updates but at a less frequent interval?

  • REST API: various null fields violate netcup-defined OpenAPI spec

    I'm guessing there will likely be other endpoints that violate spec like this, and the API should be reviewed top to bottom.

    I'm trying to implement rdns routines:

    Code
    $ go run cmd/netcup/main.go rdns delete ...

$ go run cmd/netcup/main.go rdns get ...
Error: decode response: decode application/hal+json: decode RdnsIpv4: callback: decode field "rdns": unexpected byte 110 'n' at 8
exit status 1

    When I look at the actual response directly:

    Code
    $ curl -s -H "Accept: application/hal+json" \
  -H "Authorization: Bearer $(go run cmd/netcup/main.go auth print-access-token)" \
  -X 'GET' \
  'https://servercontrolpanel.de/scp-core/api/v1/rdns/ipv4/...' | jq
{
  "rdns": null
}

    This is a problem because the OpenAPI spec does not say this is possible.

    Code
    $ jq '.components.schemas.RdnsIpv4' openapi.json
{
  "type": "object",
  "properties": {
    "rdns": {
      "type": "string"
    }
  }
}

    The spec is missing a nullable which you can read about here: https://swagger.io/docs/specifica…ata-types/#null

    Because the string data type is missing a nullable attribute, OpenAPI generator tools, like ogen in my case, generates:

    Code
    type RdnsIpv4 struct {
    Rdns OptString `json:"rdns"`
}

    Which only allows this field to be a string or absent, but not null.

  • REST API: looking for clarification on default query limit behavior

    I'd like to gain clarity on default query limit behavior. For example, I'm currently playing with the `GET /api/v1/servers` endpoint.

    The OpenAPI spec defines a `limit` query param with minimum: 0, but doesn't specify a default value. I only have a few servers, so:

    • What happens when `limit` is not provided? Does the API return all servers, or is there a server-side default that should be documented?
    • Is there a maximum allowed value for limit?
    • How do we ensure pagination is idempotent? It may be possible after a limit(+offset) that the query results can change, resulting in missed servers on the next page limit+offset(+offset).

    It would be helpful if these defaults/limits could be documented in the OpenAPI spec (via default and maximum in the schema).

  • REST API: session management: allow users to review and log out active sessions, especially because sessions remain active even after password changes

    The API documentation only lists the `/revoke` endpoint, which only revokes a specific JWT. The OIDC realm also makes available the `/logout` endpoint, which allows revoking a session entirely.

    However, being able to revoke a specific token, or logout a current session, requires having said tokens. If the user discards/forgets these tokens, but believes them to be compromised or simply wishes to revoke/logout unused sessions, they can only wait 30 days of disuse before those refresh tokens fall off.

    Users should be allowed to review their active sessions and log them out. Since this is Keycloak, I believe the appropriate areas that we can not access that we should are:

    This is especially important as I just discovered that when I change my SCP password, previous refresh tokens still remain active. Lets say I were compromised by an attacker who takes control of my netcup resources through the REST API. Even if I change my password, I would still be compromised.

    Along with access to session management, users should be able to revoke all sessions at once as an optional setting during a password change.

  • REST API: Keycloak OAuth 2.0 still allows grant_type=password disallowed by OAuth 2.0 Security BCP (and other BCP MUST violations)

    I am able to generate perpetual refresh tokens using legacy username/password authentication and without use of any 2FA:

    OAuth 2.1 does not have this grant type, and OAuth 2.0 Security BCP (Best Current Practice) disallows password grants:

    Zitat von https://datatracker.ietf.org/doc/html/rfc9700#name-resource-owner-password-cre

    2.4. Resource Owner Password Credentials Grant

    The resource owner password credentials grant [RFC6749] MUST NOT be used. This grant type insecurely exposes the credentials of the resource owner to the client. Even if the client is benign, usage of this grant results in an increased attack surface (i.e., credentials can leak in more places than just the authorization server) and in training users to enter their credentials in places other than the authorization server.

    Furthermore, the resource owner password credentials grant is not designed to work with two-factor authentication and authentication processes that require multiple user interaction steps. Authentication with cryptographic credentials (cf. WebCrypto [W3C.WebCrypto], WebAuthn [W3C.WebAuthn]) may be impossible to implement with this grant type, as it is usually bound to a specific web origin.

    In addition, while reviewing the OIDC configuration, there were several other "MUST" violations as well as some other settings that should follow the "SHOULD" recommendations. It may be worth performing a compliance review of your configuration against the security BCP. Issues that I immediately see:

    • PKCE `plain` method exposes verifier, PKCE is optional and not mandatory
    • `implicit` grants

  • REST API: OIDC token refresh_token response includes new refresh token when it should not

    In the documentation for authentication, netcup states:

    Zitat

    Refresh access token

    • The offline refresh token can be used multiple times and does not expire as long as it is used at least once every 30 days.
    Code
    curl '.../realms/scp/protocol/openid-connect/token' \
  -d 'client_id=scp'\
  -d 'refresh_token=<refresh_token>' \
  -d 'grant_type=refresh_token'  | jq

    The documentation wording makes me believe netcup's intent is that the original refresh token continues to be valid in perpetuity as long as it is used at least once every 30 days. That should mean it is not necessary to issue a new refresh token in the response for this endpoint. See:

    Zitat von https://www.oauth.com/oauth2-servers/access-tokens/refreshing-access-tokens/

    The response to the refresh token grant is the same as when issuing an access token. You can optionally issue a new refresh token in the response, or if you don’t include a new refresh token, the client assumes the current refresh token will continue to be valid.

    and

    Zitat von https://datatracker.ietf.org/doc/html/rfc6749#section-1.5

    (H) The authorization server authenticates the client and validates

    the refresh token, and if valid, issues a new access token (and,

    optionally, a new refresh token).

    However, this endpoint provides a response that includes the following key-values, including a new `refresh_token` which does not match behavior expected by documentation:

  • Route GET /api/v1/openapi

    Zitat von [netcup] Sarah G.

    In einem ersten Schritt haben wir nun die Version des SCP in die OpenAPI Spec eingefügt.

    Die Version wird bei jedem SCP Release angepasst, unabhängig davon, ob es eine Änderung an der API gegeben hat oder nicht.

    Nach der Beta-Phase wird es keine breaking changes mehr an der API gegeben. Bei erforderlichen breaking changes erfolgt die Versionierung einzelner Endpoints über den Pfad.

    Das Thema Changelog haben wir weiterhin am Radar.

    Please include this important lifecycle management information somewhere in the Swagger documentation, such as in the description field:

  • REST API: Keycloak "ID" can be trivially confused with Keycloak "preferred_username" which is netcup "Customer number"

    Prior to implementation of this new REST API using Keycloak for IAM, the only identity customers needed to be aware of was their netcup customer number, used in CCP/SCP.

    However, with Keycloak, we now have to be aware that the customer number is being used as a `username` and that there is a separate `id` unrelated to the customer number.

    If possible, the customer number/username should stay in lockstep with id. And for the time being, this distinction should be clarified in documentation.