Technical Program Manager (Data Annotation, NLP) (offline)

Skills:
Strong people skills; seamlessly work across disciplines
1. Build long-lasting relationships with external parties to guide the future direction of human evaluation
-Some experience with contract negotiation, security controls.
2. Strong technical aptitude, with experience and knowledge of training data generation for machine learning purposes.
-Required experience with training and evaluation data generation for text processing (NLP)
-Experience with a range of common data and file formats, general knowledge of AWS ecosystem
-Experience performing annotations and developing reference “golden” datasets
3. Oozes comfort and confidence with data, especially in the face of disparate data sets
-Experience handling and managing content with sensitive data containing PII (personally identifiable
information).
4. Ability to break down complex tasks for consumption by a diverse, not immediately technical audience


Responsibilities:
-Establish a human annotation process at , combining the services of internal annotators with those of external annotation vendors;
-Act as the process flywheel to ensure evaluation tasks are prioritized, loaded, and worked on; drive evaluation quality and throughput;
-Define guidelines and user interfaces for human evaluation tasks in close collaboration with the Product team;
-In close collaboration with internal and external resources, monitor and report on the quality of the data generated through the human annotation process;
-Conduct vendor evaluations; provide ongoing recommendations to the Product team for the human evaluation vendor and platform of choice;
-Generate data sets for measurement and experimentation; source external data as required
Manage the human evaluation budget.

About Data Science UA

We understand that you need more than just a search engine to find IT and technical professionals or to progress your own career, that is why we established Data Science UA.
One place connecting business and developers.

Company website:
https://data-science-ua.com/

DOU company page:
https://jobs.dou.ua/companies/data-science-ua/