Currently OpenVSCode Server is released with support for x86_64 architectures only. Thankfully though it's quite easy to get it going on a Rasberry Pi (4b in my case).

1. Login to your Rasberry Pi. I created a bin folder in my home directory to store Node.js, but anywhere should work.
2. Download the Node.js binaries (wget https://nodejs.org/dist/v14.18.1/node-v14.18.1-linux-arm64.tar.xz). This fetches the latest LTS version.
3. Uncompress the archive - tar xvf node-v14.18.1-linux-arm64.tar.xz.
4. Next I updated $PATH so it would make calling node and npm easier. I did this by editing my ~/.bashrc file - export PATH="/home/jimmy/bin/node-v14.18.1-linux-arm64/bin:$PATH".
5. Next I downloaded the latest OpenVSCode Server release - wget https://github.com/gitpod-io/openvscode-server/releases/download/openvscode-server-v1.61.0/openvscode-server-v1.61.0-linux-x64.tar.gz and uncompressed it - tar zxvf openvscode-server-v1.61.0-linux-x64.tar.gz`.
6. You can either remove the node_modules folder inside the openvscode-server-v1.61.0-linux-x64 folder or just move it aside. I moved it aside - mv node_modules/ node_modules.old/.
7. Now run npm install and it will reinstall the necessary Node.js modules. This may take a minute or two.
8. Once the modules have been installed run - node ./out/server.js.

That's it! You should see something like:

[jimmy@my.raspberry.pi ~/openvscode-server/openvscode-server-v1.61.0-linux-x64]$ node ./out/server.js
[main 2021-10-22T19:50:06.927Z] Web UI available at http://localhost:3000

I don't know if this is the best way to get OpenVSCode Server running on a Raspberry Pi, but it was the quickest and easiest way I could find! Ideally the Gitpod team updates their GitHub workflow (assuming that's what they're using to create releases) to build a release for arm64, I can't imagine it would be too difficult.