Job Description/Requirements
What you will be doing:
- Measure and benchmark model and application performance.
- Analyze model accuracy and bias and recommend the next course of action & improvements.
- Drive the gathering, building, and annotation of domain specific datasets to train LLMs for different tasks and applications.
- Characterize performance and quality metrics across platforms for various AI and system components.
- Participate in developing and reviewing code, design documents, use case reviews, and test plan reviews.
- Help innovate, identify problems, recommend solutions and perform triage in a collaborative team environment and collaborate with various teams on new tool features and improvements of existing tools.
What We Need To See:
- Master’s degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math
- 12 + years of experience.
- Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
- Solid understanding of ML/DL techniques, algorithms and tools with exposure to CNN, RNN (LSTM), Transformers (BERT, BART, GPT/T5, LLMs)
- Hands-on experience on conversational AI Technologies like Natural Language Understanding, Natural Language Generation, Dialog systems (including system integration, state tracking and action prediction), Information retrieval and Question and Answering, Machine Translation etc.
- Experience with Training BERT, GPT for different NLP and dialog system tasks using “PyTorch” Deep Learning Frameworks and performing NLP data wrangling and tokenization.
- Develop large scale multimodal information retrieval system leveraging open source frameworks such as LlamaIndex, LangChain, ElasticSearch and so on.