Datasets with loading scripts - no longer supported

[baberabb]

As of datasets==4.0, loading scripts and trust_remote_code are no longer supported. Started this thread to track tasks which still rely on the scripts so that the datasets can either be updated upstream in HF or in the task configs.

Just add a comment, if anyone comes across any.

[baberabb]

afrobench's masakhapos: masakhane/masakhapos

[Avelina9X]

Gone through all the a*** groups. Found remote code, missing datasets, an incorrect name, and some datasets marked as remote code required but have been fixed on the hub to work without code. I may have missed some, but this is what I found so far.

aclue: tyouisen/aclue (remote code)

aexams: Hennara/aexams (remote code)

afrobench:

• adr: masakhane/diacritics-restoration (missing)
• afriqa: masakhane/afriqa-gold-passages (remote code)
• afrisenti: masakhane/afrisenti (fixed on hub, task needs updating)
• flores: facebook/flores (remote code)
• mafand: masakhane/mafand (fixed on hub, task needs updating)
• masakhaner: masakhane/masakhaner-x (remote code)
• masakhapos: masakhane/masakhapos (remote code)
• nollysenti: Davlan/nollysenti (fixed on hub, task needs updating)
• ntrex: masakhane/ntrex_african (fixed on hub, task needs updating)
• salt: Sunbird/salt (fixed on hub, task needs updating)
• xlsum: csebuetnlp/xlsum (fixed on hub, task needs updating)

anli: anli (should be facebook/anli)

arab_culture_completion: boda/arabic_cluture (missing)

asdiv: EleutherAI/asdiv (remote code)

[baberabb]

super-glue: added full path in #3169

[BKHMSI]

mgsm: juletxara/mgsm also has a similar issue!

[fzyzcjy]

mmlu has same issue

[KyleMylonakisProtopia]

Hey all, is there any current plan to support datasets 4.0 and grater beyond what is discussed in this issue? If we don't run the tests outline above, can the pin at least be removed and moved to some kind of raised error on the remaining tests which have issues?

[jannalulu]

longbench v1 zai-org/LongBench still uses remote code, would not load without trust_remote_code=True

[zoxxxx]

This problem also occurs with race.

[baberabb]

updated the race dataset to parquet. btw also discovered the datasets-cli utility, which I didn't even know existed but apparently been around for awhile. datasets-cli convert_to_parquet EleutherAI/race was pretty quick

[upunaprosk]

Same issue with crows_pairs