Why am I running into issues creating Kubernetes Persistent Volume Claims?

Linode Staff

I'd really like to create and start working with a Persistent Volume for my pod(s), but I keep running into errors!

2 Replies

Linode Staff

A great question! If you haven't already, I'd recommend taking a look through our official guide on the matter -- even if you have, I'd still recommend reading through it once more to ensure that you've formatted your PVC manifest correctly. If you've already done so, we can move on to the following troubleshooting steps.

Identifying the Underlying Issue


In our time working with Persistent Volumes and Persistent Volume Claims, we have run into the following two error messages:

  1. "reason": "Your OAuth token is not authorized to use this endpoint."
  2. Failed to set up metadata service: couldn't initialize Linode client: instance not found

In order to determine the particular error that you're running into when attempting to create a PVC you can run the following command:

kubectl logs -n kube-system csi-linode-controller-0 linode-csi-plugin

This command queries the logs for the CSI controller plugin, and will display any errors being returned by the API. Does the output you receive match up with either of the above error messages? If so, excellent! Skip ahead to the next section for a couple of solutions we've identified for these errors.

If not, we'd encourage you to include your error message and any other relevant information in a comment below -- we on the Linode staff - as well as other Community Site members - would be glad to help out.

Resolving the Issue


Once you've identified the specific error you're running into, we can move forward with resolving the issue.

Error Message #1:

"reason": "Your OAuth token is not authorized to use this endpoint"

This is likely related to API key permissions. To elaborate further, your API key must have read/write permissions for Volumes and Linodes. If your API key does not have the above permissions, you will need to generate a new API key, and re-enter it into your Kubernetes master node.

Error Message #2:

Failed to set up metadata service: couldn't initialize Linode client: instance not found

If you're confident that your API key is set up correctly, then it's likely that this is related to your Linode's label -- to clarify, in order for the Container Storage Interface (CSI) plugin to function, your Linode's label needs to be lowercase. If your Linode's label includes any uppercase letters, you'll need to rename your Linode by selecting it in the Cloud Manager, clicking on your Linode's label, and removing any uppercase letters in the text box that appears.


As mentioned above, this is by no means an exhaustive list of errors you might come across when attempting to create a PVC -- if you happen upon an error message not included in this post, feel free to post it as a comment below!

I am getting this error:

Warning FailedMount 15s (x454 over 15h) kubelet, lke21851-27625-6051768b526a MountVolume.MountDevice failed for volume "pvc-83961bb5e1f44bde" : rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /var/lib/kubelet/plugins/linodebs.csi.linode.com/csi.sock: connect: connection refused

I have a different PVC the same size attached to the same node and mounted to a different pod on the same node, that has been working for a few months. The configuration is the same except for a different image on the pod and a different name on the volume. I have tried rotating the access token, the PVC gets bound successfully but the pod can't mount it.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct