Skip to content

Conversation

@bdewilde
Copy link
Collaborator

Description

  • Added UDHR dataset, a collection of (~200) translations of the Universal Declaration of Human Rights

Motivation and Context

I wanted a relatively lightweight and comprehensive set of source documents for getting language-specific distributions of characters, for use in certain data augmentation transform functions. The current solution relies on ConceptNet, which is 1. very large and 2. less comprehensive, thus 3. unsatisfactory.

How Has This Been Tested?

Wrote the usual tests, and they all pass. Fixed some other datasets' buggy tests in the process!

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation, and I have updated it accordingly.

@bdewilde bdewilde merged commit 17333a0 into develop Aug 30, 2019
@bdewilde bdewilde deleted the feature/add-udhr-dataset branch August 30, 2019 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants