Problem with nuxt imports: Vitest Coverage stays at 100% for components regardless of the existence of any component test because of nuxt imports

[FlorianPhilipp2007]

Environment

• Operating System: Darwin
• Node Version: v18.16.0
• Nuxt Version: 3.11.2
• CLI Version: 3.11.1
• Nitro Version: 2.9.6
• Package Manager: [email protected]
• Builder: -
• User Config: devtools, typescript, extends, components, ssr, runtimeConfig, app, css, build, modules, experimental, i18n, proxy, vite, public, nitro, hooks
• Runtime Modules: @pinia/[email protected], [email protected], [email protected], @nuxt/test-utils/[email protected], @nuxtjs/[email protected]
• Build Modules: -

Reproduction

https://stackblitz.com/edit/nuxt-starter-gvfsue?file=nuxt.config.ts,components%2Fnested%2FTestNestedUntested.vue

vitest --coverage is not working in StackBlitz. Please download the project and try it locally.

Describe the bug

Description:
When the components option in the Nuxt configuration is set to true (enabling automatic global registration of components), the test coverage report shows 100% coverage for all components, regardless of whether tests actually exist for them. Something is importing the untested TestNestedUntested. V8 is reporting coverage for it. Maybe Nuxt imports all those files.

Actual Behavior:
The coverage report incorrectly shows 100% coverage for all components, including those without any tests like TestNestedUntested.vue in component directory. Seems to be a problem caused by nuxt importing all files.

Expected Behavior:
The coverage report should accurately reflect the coverage for each component. Only TestNested.vue should show 100% coverage, while TestNestedUnested.vue should show 0% coverage or appropriate coverage based on the absence of tests.

Additional context

No response

Logs

No response

[stackblitz]

Start a new pull request in StackBlitz Codeflow.

• nuxt-starter-gvfsue (Open reproduction)

[danielroe]

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

[dosubot]

Hey @FlorianPhilipp2007! I'm here to help you with any bugs, questions, or contributions. Let's work together to find a solution!

To resolve the issue where Nuxt imports are causing Vitest coverage reports to incorrectly show 100% coverage for all components, you need to adjust the coverage configuration in your vitest.config.ts file. Specifically, you should ensure that the coverage configuration accurately reflects the files and directories you want to include and exclude from the coverage report.

Here's an updated version of your vitest.config.ts file with a more precise coverage configuration:

import { resolve } from 'pathe'
import { configDefaults, coverageConfigDefaults, defineConfig } from 'vitest/config'
import { isWindows } from 'std-env'

export default defineConfig({
  resolve: {
    alias: {
      '#build/nuxt.config.mjs': resolve('./test/mocks/nuxt-config'),
      '#internal/nuxt/paths': resolve('./test/mocks/paths'),
      '#build/app.config.mjs': resolve('./test/mocks/app-config'),
      '#app': resolve('./packages/nuxt/dist/app'),
    },
  },
  test: {
    globalSetup: './test/setup.ts',
    setupFiles: ['./test/setup-env.ts'],
    coverage: {
      exclude: [...coverageConfigDefaults.exclude, 'packages/nuxt/src/app', 'playground', '**/test/', 'scripts', 'vitest.nuxt.config.ts'],
    },
    testTimeout: isWindows ? 60000 : 10000,
    exclude: [...configDefaults.exclude, 'nuxt/**', '**/test.ts', '**/this-should-not-load.spec.js'],
    poolOptions: {
      threads: {
        maxThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
        minThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
      },
    },
  },
})

In this configuration:

• The resolve.alias option is used to mock certain Nuxt paths, ensuring that the correct files are referenced during testing.
• The coverage.exclude option is set to exclude specific directories and files from the coverage report, including the packages/nuxt/src/app directory and test-related files.
• The test.exclude option is set to exclude certain files and directories from being tested, ensuring that only the relevant files are included in the coverage report.

This should help ensure that the coverage report accurately reflects the tested and untested components ^[1][2].

To continue talking to Dosu, mention @dosu.

[FlorianPhilipp2007]

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

I first raised the issue at vitest but they think its a nuxt problem:
vitest-dev/vitest#6029 (comment)

I tried the exclude from .nuxt but it doesen't help

[danielroe]

It seems like the issue is the one Ari linked - it's a Vue sourcemap bug: vitest-dev/vitest#3607 (comment).

[AriPerkkio]

This report contains two issues:

1. Something (nuxt ?) is importing files that are not being tested. Node's V8 coverage includes every file that is loaded via Node. The repro is testing TestNested.vue, but is also unexpectedly seeing coverage for UntestNested.vue. Vitest is not importing UntestNested.vue.
2. Coverage is incorrectly 100%. This is caused by Vue's sourcemaps being off. We've seen so many of these issues that I think I'll need to raise an issue on Vue some day.

[DenisLug]

@AriPerkkio, @danielroe: Thank you both for your help so far.

The issue with the coverage report has evolved after updating vitest to version 2 (see comment in vitest issue, Stackblitz demo). Previously, the coverage incorrectly showed 100% for all components, but now it shows 0% coverage for files that do have tests.

To summarize the current status:

• With the istanbul provider, the coverage report collects only the coverage for the utils directory
• With the v8 provider, the coverage report now shows 0% coverage for files that have tests

Given these changes, could we clarify who will be investigating this further? Is this something that should be addressed by the Nuxt team, or should we continue the discussion with the Vitest team?