elwyn-gitlab-lsp-test

GitLab Language Server - TypeScript (Experiment)

The GitLab Language Server provides a common interface for other IDE extensions to build out GitLab functionality. The language server supports:

• GitLab Duo Code Suggestions
• GitLab Duo Chat (coming soon)

For bugs and feature requests, open a new issue.

[[TOC]]

Introduction

LSP (Language Server Protocol) is a technology that provides an abstraction layer between tools that provide analysis results (language servers) and the "consumer" IDEs (language clients). It provides a generic interface to provide analysis results in LSP-enabled IDEs. Implement the analysis one time, and all IDEs then benefit.

This project is an LSP-based language-server that serves GitLab AI Code Suggestions to LSP-enabled IDEs.

The server supports LSP protocol version 3.17.

This language server leverages the textDocument/completion LSP method to serve code suggestions. For most editors, this hooks into the IntelliSense feature, which you can invoke with the Ctrl + space Ctrl + N shortcuts.

Run the LSP

The LSP client should be responsible for starting the server process and setting up the connection. The LSP server supports multiple communication methods, and you must provide one of them at startup:

• IPC - Only for communication with the Node process. The server should be a Node app. The server is not published as an NPM package, but you can use the out/node/main.js module if you've compiled the project locally for testing purposes.

  node 'out/node/main.js' --node-ipc
  

• TCP Socket - Provide a socket port when starting the server process. The client should be waiting on that port for the server to connect.

  /opt/gitlab-lsp --socket=6789
  

• STDIO

  /opt/gitlab-lsp --stdio
  

• Named Pipes - The Language Server listens as a named pipe server for the client to connect.

  /opt/gitlab-lsp --pipe=pipeName
  

Check main.ts to see how the vscode-languageclient Node JS library handles server startup.

Use open tabs as context

For better results from GitLab Duo Code Suggestions, ensure that Open Tabs Context is enabled in your IDE settings. This feature uses the contents of the files currently open in your IDE, to get more accurate and relevant results from Code Suggestions. Like prompt engineering, these files give GitLab Duo more information about the standards and practices in your code project.

To get the most benefit from using your open tabs as context, open the files relevant to the code you want to create, including configuration files. When you start work in a new file, Code Suggestions offers you suggestions in the new file.

Prerequisites:

• Requires GitLab 17.1 or later. Earlier GitLab versions that support Code Suggestions cannot weight the content of open tabs more heavily than other files in your project.
• Requires version 4.14.2 or later of the GitLab Workflow extension for Visual Studio Code.
• GitLab Duo Code Suggestions must be enabled for your project, and configured in the GitLab Workflow plugin.
• Requires a supported code language.

1. Download and install a supported version of the GitLab extension from the Visual Studio Marketplace. For more about configuring the extension, see its setup instructions.
2. Open the files you want to provide for context. Advanced Context uses the most recently opened or changed files for context. If you don't want a file sent as additional context, close it.
3. To fine-tune your Code Generation results, add code comments to your file that explain what you want to build. Code Generation treats your code comments like chat. Your code comments update the user_instruction, and then improve the next results you receive.

As you work, GitLab Duo provides code suggestions that use your other open files (within truncation limits) as extra context.

To learn about the code that builds the prompt, see these files:

• Code Generation: ee/lib/api/code_suggestions.rb in the gitlab repository
• Code Completion: ai_gateway/code_suggestions/processing/completions.py in the modelops repository

We'd love your feedback about the Advanced Context feature in issue 258.

Advanced Context supported languages

The Advanced Context feature supports these languages:

• Code Completion: all configured languages.
• Code Generation: Go, Java, JavaScript, Kotlin, Python, Ruby, Rust, TypeScript (.ts and .tsx files), Vue, and YAML.

Install the language server client

To install the language server binary locally:

1. Download the language server binary from the Package Registry page. For every release you can download the binary that matches your OS and architecture. In the list of assets linked in the release, binaries end with the string <os>-<arch>.

2. Place the binary anywhere on your file system. The configuration in this documentation assumes you've placed the binary under /opt/gitlab-lsp.

3. The language server requires an access token to authenticate with the GitLab API. Create a personal access token (PAT) with the api scope, or OAuth token with the same scope. Provide this token to the server sending the didChangeConfiguration notification from the client to the server after the server is started. See the details here.

4. In your terminal, make the binary executable with this command:

   chmod +x opt/gitlab-lsp
   

5. The client implementation depends on your text-editor or IDE. See the next section.

Allow the binary to run on macOS

MacOS users might encounter a message in the terminal, saying the binary was killed as soon as you started it. This happens because of a security feature called Gatekeeper. To allow the binary to run, clear its extended attributes and ad-hoc sign it, which establishes trust with Gatekeeper.

Run these commands in your terminal, and enter your password if prompted:

sudo xattr -c opt/gitlab-lsp
codesign --force -s - opt/gitlab-lsp

LSP client configuration

• VS Code

This section is still in progress...

Node version

We use the Node version defined in the Microsoft/vscode project. We must make sure to keep the major version the same as what the pkg-fetch module supports.

The pkg-fetch module provides the node version used by pkg when packaging the language server. Each version of pkg-fetch has a static set of node builds. You can look up the versions on the pkg-fetch releases page of the GitHub project.

Prerequisites:

• Make sure the major version is the same for the pkg and the development versions.

To upgrade the version of Node in use:

1. Update pkg version, if there is a new one:
   1. Upgrade the pkg-fetch module by editing the package.json file's overrides section manually.
      • Note: To get the updated package, make sure to rm -rf node_modules and run npm install.
   2. Check the pkg-fetch releases page for the Node version. Verify macos-arm64 builds exist for the selected version.
   3. Update .gitlab-ci.yml (test-pkg-node) with the new version number.
2. Update the development/VS Code version:
   1. Update .tool-versions file with the new version number.
   2. Update .gitlab-ci.yml (default) with the new version number.
   3. Update src/tests/int/node_version.test.ts with the correct version number.
   4. Update package.json scripts section where we reference the target Node version, like this: --target=node18.17.
3. Add a changelog entry.

Development setup

To debug or add new features to the language server:

1. Clone this repository.
2. Run npm install.
3. Compile the server code with npm run compile or npm run watch to compile when the changes are made.

Watch mode

Watch mode rebuilds the project automatically with every file change. Optionally, it can also update gitlab-lsp dependency in VS Code. For details, see the watch mode script.

This example uses VS Code. Run this command:

npm run watch -- --editor=vscode

Then, in VS Code, you can restart your extension development host, and it receives the latest server changes.

Script parameters

These parameters are optional:

• -e, --editor <type> - Specify editor type (e.g., vscode). If set to 'vscode', the script will copy the build files to automatically update gitlab-lsp in your VS Code Extension.
  • note: You do not need to restart the extension development host if you wish to restart the server with your changes. Simply run the "Restart Language Server" command in the command palette.
• -vp, --vscode-path <path> - Path to VSCode extension
• -p, --packages <packages...> - Specify packages to watch. If not provided, all packages will be built.

Supported editors:

• vscode

Example usage:

npm run watch -- --editor vscode --vscode-path ../my-vscode-extension --packages package1 package2

This will watch for changes, build only the specified packages, and copy the results to the specified VS Code extension path.

Start the server with test VS Code extension

You should test the LSP server together with the test LSP client. See the test LSP client (VS Code extension) documentation for how to configure the test client to run and start the server.

Start with another client, or without the client

To start the server separately, either:

• Run npm run debug.

• Run this command:

  node --inspect=6010 ./out/node/main.js --socket=6789
  

Your client should establish the socket connection, and wait on port 6789 for the server to connect.

Debug the server

To debug the Language Server, you can:

• Start the Language Server.
• Connect to the Language Server in the VS Code extension.
• Connect to the Language Server using Chrome Developer Tools.

Start the Language Server

1. Open the project in VS Code.
2. Run the Start server launch task.

The launch task starts a Language Server that listens on port 6789 for a client connection, and connects VS Code debugger to it.

Connect to LS in the VS Code extension

Prerequisites:

• If connecting with the VS Code extension, you must have the code command configured. See Launching VS Code from the command line for more information.

1. Set up the extension project for LS development.

2. Start the VS Code extension in development mode.

3. Use the command line to open this project in VS Code, replacing the path with the path of your VS Code Extension project:

   GITLAB_WORKFLOW_PATH=/Users/tomas/workspace/gitlab-vscode-extension code .
   

   This command is important. It maps the bundled sources in the extension with the source code in this project.

4. Run the Attach to VS Code Extension launch task.

Connect to LS with Chrome Developer Tools

Prerequisites:

• Your server must be a Node module, not an executable.

1. Either:
   • Start the VS Code extension in development mode, or
   • Start the server.
2. Open chrome://inspect in a Chrome browser. For more information and debugging tools, see Inspector Clients in the Node.js documentation.
3. Select Configure, and ensure your target host and port are listed. Use localhost:6010.
4. Select Open dedicated DevTools for Node.
5. Either add the debugger statement, or add console.log in your server code.
6. Recompile after you change the code.

When you start the Language Server, the inspector pauses on the breakpoints.

Test changes in downstream project pipelines

You can install the artifact created by the build_package_for_integration job in merge request pipelines in downstream projects. This allows you to run CI/CD for the downstream project without waiting for a new Language Server version to be published.

For instance, in the VS Code extension, you can run:

npm install <URL>

where URL is the URL reported in the job log. Then, you can commit the package*.json changes and push them to a merge request.

Lint documentation

This project follows the documentation guidelines of GitLab and uses markdownlint and vale to improve the content added and make sure it adheres to our guidelines. You can find information on installing the linters or integrating with your editor in the GitLab documentation.

Bundling

We need to create a self-contained bundled JavaScript file that is started by VS Code. We use esbuild to bundle all the dependencies into one file located in out/node/main-bundle.js.

Packaging

Run:

npm run package

Your binary is available in the bin folder of the project.

Releases

Releases consist of deploying an NPM package and a corresponding generic package to the package registry.

To release a new version of the packages:

1. Create a new branch git checkout -b 2023-11-06-release

2. Run npm version major/minor/patch. Use major, minor, or patch as defined by semantic versioning.

   • This command creates a new version, updates the changelog, and tags the commit.

3. Push the branch and the tag. For example, if you created v3.4.5, run this command:

   git push && git push origin v3.4.5
   

4. Create a merge request from the 2023-11-06-release branch:

   • Use the version number in the merge request name, like this: Release LS version 3.4.5.
   • Make sure the merge request is NOT set to squash, because squashing changes the release commit SHA.

5. No review is needed, as no code has changed.

6. Merge the Release LS version 3.4.5 merge request.

Communicate updates to downstream extensions

Communicate updates to the downstream projects through integration epics. An epic contains upgrade issues for each internal project that consumes the Language Server. Example.

Once per week, if any new releases after the last integration epic would benefit the downstream projects, create a new integration epic.

Troubleshooting

Advanced Context feature and memory

The Advanced Context feature stores file information in memory. If you encounter memory errors, use fewer files to generate context, and close any you don't need.