Avoid parsing joint rule codes as distinct codes in # noqa

[charliermarsh]

Summary

We should enable warnings for unsupported codes, but this at least fixes the parsing for # noqa: F401F841.

Closes #12808.

[InSyncWithFoo]

is_alphanumeric() also allows non-ASCII-digits and letters:

# noqa: A三

Did you mean to use is_ascii_alphanumeric() instead?

[charliermarsh]

Seems reasonable to exclude non-ASCII.

[codspeed-hq]

CodSpeed Performance Report

Merging #12809 will not alter performance

_{Comparing charlie/parse (c2e9746) with main (f837428)}

Summary

✅ 32 untouched benchmarks

[InSyncWithFoo]

On that same note, is_whitespace() and trim_start() / trim_end() also handle Unicode whitespace. They should be replaced with is_ascii_whitespace() and trim_ascii_start() / trim_ascii_end(), accordingly.

"ASCII whitespace" is defined as "space, tab, LF, FF, CR". In our case, since Python comments don't allow line breaks, that boils down to just spaces and tabs.

[github-actions]

ruff-ecosystem results

Linter (stable)

ℹ️ ecosystem check detected linter changes. (+3 -0 violations, +0 -0 fixes in 2 projects; 52 projects unchanged)

apache/airflow (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --no-preview --select ALL

+ providers/src/airflow/providers/apache/hdfs/sensors/hdfs.py:45:37: RUF100 [*] Unused `noqa` directive (unknown: `Ignore`)
+ providers/src/airflow/providers/apache/hdfs/sensors/hdfs.py:50:38: RUF100 [*] Unused `noqa` directive (unknown: `Ignore`)
+ providers/src/airflow/providers/google/cloud/hooks/bigquery.py:59:42: RUF100 [*] Unused `noqa` directive (unknown: `Used`)

pandas-dev/pandas (+0 -0 violations, +0 -0 fixes)

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
RUF100	3	3	0	0	0

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+3 -0 violations, +0 -0 fixes in 1 projects; 53 projects unchanged)

apache/airflow (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ providers/src/airflow/providers/apache/hdfs/sensors/hdfs.py:45:37: RUF100 [*] Unused `noqa` directive (unknown: `Ignore`)
+ providers/src/airflow/providers/apache/hdfs/sensors/hdfs.py:50:38: RUF100 [*] Unused `noqa` directive (unknown: `Ignore`)
+ providers/src/airflow/providers/google/cloud/hooks/bigquery.py:59:42: RUF100 [*] Unused `noqa` directive (unknown: `Used`)

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
RUF100	3	3	0	0	0

[charliermarsh]

But Python comments can contain non-ASCII whitespace? We already have an is_python_whitespace that respects the spec for tokens: https://docs.python.org/3/reference/lexical_analysis.html#whitespace-between-tokens

[InSyncWithFoo]

But Python comments can contain non-ASCII whitespace?

Fair enough, though I still don't see why someone would want to use non-ASCII whitespace in the noqa part of a comment. A user might want to write an explanation in their native language in the same comment, sure, but noqa and the rule codes are never localized.

[InSyncWithFoo]

One more thing: ParsedFileExemption::try_extract() currently treats A001 , B002 (whitespace before comma) and A001,,B002 (empty slot) as if B002 doesn't exist. Is this difference in behaviour intended?

[dhruvmanila]

I think we should show a warning in this case but still parse the remaining codes (if that's possible currently). This would be similar to how we recover from a missing element in list parsing: https://play.ruff.rs/308ab356-9355-43f3-99b7-12c7c2de7334.