Semantic highlighting broken (JDT "Request textDocument/semanticTokens/full failed.")

[emilyploszaj]

I've run into a strange issue where syntax highlighting only contains keywords and primitives, rather than the rich semantic tokens I'm used to. Looking in the log, the request for textDocument/semanticTokens/full to JDT is failing for some reason over and over again. I've tested this in several versions of vscode over the last year, using several different versions of this extension, because it definitely was working relatively recently. I've also tested several versions of Java and the issue persists in all situations.

Environment

• Operating System: Linux
• JDK version: Several (OpenJDK 17.0.8.1 primarily)
• Visual Studio Code version: Several (1.82.2 primarily)
• Java extension version: Several (Both latest release and pre-release primarily)

Relevant extension log

{
  message: '[Error - 5:35:38 PM] Request textDocument/semanticTokens/full failed.',
  level: 'info',
  timestamp: '2023-10-16 17:35:38.054'
}
{
  message: '  Message: Internal error.\n' +
    '  Code: -32603 \n' +
    'java.util.concurrent.CompletionException: java.lang.NullPointerException: Cannot invoke "org.eclipse.jdt.core.dom.CompilationUnit.getJavaElement()" because "root" is null\n' +
    '\tat java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)\n' +
    '\tat java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)\n' +
    '\tat java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:649)\n' +
    '\tat java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)\n' +
    '\tat java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)\n' +
    '\tat java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)\n' +
    '\tat java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)\n' +
    '\tat java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)\n' +
    '\tat java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)\n' +
    'Caused by: java.lang.NullPointerException: Cannot invoke "org.eclipse.jdt.core.dom.CompilationUnit.getJavaElement()" because "root" is null\n' +
    '\tat org.eclipse.jdt.ls.core.internal.handlers.SemanticTokensHandler.getAst(SemanticTokensHandler.java:84)\n' +
    '\tat org.eclipse.jdt.ls.core.internal.handlers.SemanticTokensHandler.full(SemanticTokensHandler.java:50)\n' +
    '\tat org.eclipse.jdt.ls.core.internal.handlers.JDTLanguageServer.lambda$53(JDTLanguageServer.java:1166)\n' +
    '\tat org.eclipse.jdt.ls.core.internal.BaseJDTLanguageServer.lambda$0(BaseJDTLanguageServer.java:87)\n' +
    '\tat java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)\n' +
    '\t... 6 more\n',
  level: 'info',
  timestamp: '2023-10-16 17:35:38.054'
}

Example missing advanced highlighting

[emilyploszaj]

I have no idea how this happened, but I figured out the problem for anyone with a similar issue. For some reason, it was my theme. It's important to mention that this was making code complete broken too, somehow during my update it duplicated one of my themes and made "Dark (Visual Studio) Visual Studio Dark" which looks identical to "Dark+ Default Dark+" except it breaks JDT somehow. I really don't want to know how a broken theme was created during an update and how it was breaking my semantic highlighting, but either way, I've resolved it.