go1.21: load rules: parse rules file: typechecker error: ...: could not import

[ldez]

Hello,

I know it's a bit early but I started to work on go1.21 for golangci-lint.
golangci/golangci-lint#3922

And there is an issue with go-ruleguard.
I run the tests of go-ruleguard and there is the same problem:

ruleguard init error, skip ./rules.go: typechecker error: ./rules.go:6:8: could not import github.com/quasilyte/go-ruleguard/dsl (can't find import: "github.com/quasilyte/go-ruleguard/dsl")

Currently, I don't know the root cause but I want to share it with you, I think will you see the problem faster than me.

Maybe it's related to the new package initialization order, I don't know.

I already created an issue on go-critic.

go test

$ go test ./...                             
?       github.com/quasilyte/go-ruleguard       [no test files]
?       github.com/quasilyte/go-ruleguard/analyzer/testanalyzer [no test files]
?       github.com/quasilyte/go-ruleguard/cmd/ruleguard [no test files]
?       github.com/quasilyte/go-ruleguard/internal/golist       [no test files]
?       github.com/quasilyte/go-ruleguard/internal/xtypes       [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/goutil      [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/ir  [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/irprint     [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/profiling   [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/quasigo/internal/evaltest   [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/quasigo/stdlib/qfmt [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/quasigo/stdlib/qstrconv     [no test files]
?       github.com/quasilyte/go-ruleguard/ruleguard/quasigo/stdlib/qstrings     [no test files]
--- FAIL: TestAnalyzer (4.78s)
    --- FAIL: TestAnalyzer/gocritic (0.38s)
        analysistest.go:295: error analyzing ruleguard@gocritic: load rules: parse rules file: typechecker error: ./testdata/src/gocritic/rules.go:6:8: could not import github.com/quasilyte/go-ruleguard/dsl (can't find import: "github.com/quasilyte/go-ruleguard/dsl")
...

[quasilyte]

Perhaps a newer version of x/tools should be used yet again. :)
Let me try updating it here.

[quasilyte]

See #450

[ldez]

Sadly, the update of x/tools doesn't fix the problem.

[ldez]

The rc4 is here, the GA is closer than ever.

@quasilyte can I help you?

[ldez]

go1.21 is available https://go.dev/blog/go1.21

[liggitt]

go env output changed in go1.21 to be single-quoted and escape single quotes as '\'' in https://go-review.googlesource.com/c/go/+/493535

parseGoEnv() in https://github.com/quasilyte/go-ruleguard/blob/master/internal/goenv/goenv.go has to handle both double-quoted and single-quoted formats now, something like:

	} else {
		// Line format is: `$name="$value"` before go1.21
		// Line format is: `$name='$value'` in go1.21+
		for _, l := range lines {
			parts := strings.Split(strings.TrimSpace(l), "=")
			if len(parts) != 2 {
				continue
			}
			val := parts[1]
			switch {
			case val == `""` || val == `''`:
				val = ""
			case len(val) > 2 && val[0] == '"' && val[len(val)-1] == '"':
				// go < 1.21 before https://github.com/golang/go/commit/f379e78951a405e7e99a60fb231eeedbf976c108
				var err error
				val, err = strconv.Unquote(parts[1])
				if err != nil {
					continue
				}
			case len(val) > 2 && val[0] == '\'' && val[len(val)-1] == '\'':
				// go >= 1.21 after https://github.com/golang/go/commit/f379e78951a405e7e99a60fb231eeedbf976c108
				// trim leading/trailing single-quotes
				val = val[1 : len(val)-1]
				// replace internally escaped single-quotes
				val = strings.ReplaceAll(val, `'\''`, `'`)
			default:
				val = ""
			}
			vars[parts[0]] = val
		}
	}

[quasilyte]

AFAIK parseGoEnv is only used in the fallback importer, if everything else fails.
It's a good thing to make it work again, but other importers should be able to import the packages from the standard library.

[liggitt]

AFAIK parseGoEnv is only used in the fallback importer, if everything else fails. It's a good thing to make it work again, but other importers should be able to import the packages from the standard library.

not sure how it got there, but resolving this fixed go-critic/go-critic#1359 (comment)

[liggitt]

go team also suggested the following in https://go-review.googlesource.com/c/go/+/493535:

If you call go env with a specific list of variables, it outputs those variables unquoted, separated by newlines. That's probably the simplest and most portable option.

If for some reason you don't know the list of variables needed, you could call go env once, parse out the list of (unquoted) variable names, and then call go env again with those names as explicit arguments.

Or, to be robust to variables that contain internal newline characters, you could reinvoke go env once per variable — although in most cases that seems overkill. 😅

(Based on https://github.com/search?q=repo%3Aquasilyte%2Fgo-ruleguard%20goenv&type=code, it looks like go-ruleguard only needs GOROOT, GOPATH, GOARCH, GOOS, and CGO_ENABLED?)

[ldez]

The link seems private.

The public link is https://go-review.googlesource.com/c/go/+/488375

[cristaloleg]

Looks like public git with extra steps http://go.dev/cl/488375