AppTek.ai offers state-of-the-art research and development for generative pre-trained transformer (GPT) large language models (LLMs) for generation of fluent human-like text.
Generative Large Language Models (LLMs) are advanced systems designed to understand and generate human-like text based on the patterns and structures learned from vast datasets. Models are trained on diverse data sources which enable them to perform a wide range of language-based tasks such as composing essays, answering questions or creating dialogue, amongst many other applications. By leveraging AppTek.ai's data services combined with sophisticated machine learning techniques, including supervised, unsupervised, and reinforcement learning, enterprises can cost-effectively harness and deploy LLMs to generate coherent and contextually relevant text for tasks such as content creation, customer service, education, and more.
AppTek.ai's distributed workforce develop and delivers the highest quality data for the development and deployment of bespoke LLM's based on a specific use case or client domain. AppTek.ai's data services team develops multilingual prompt-pairs for LLMs with training methodology including:
AppTek.ai offers experience in pre-training instruction finetuning (IFT) and retrieval augmentation (RAG) that yield strong models which are small in size and comparable in performance compared to much larger models.
Research is underway for count-motivated word vector initialization as well as branch optimization, which both have shown to offer a performance boost for foundation model deployment at the early stage of training.
By systematically checking the gains and losses of replacing the adaptive learning rate optimization algorithm, scientists have found improvements in GPU memory resource optimization which could change common LLM training practices.
By focusing on decision boundaries between similarly ranked completions (going over adjacent completion pairs versus going over all completion pairs) and introducing a margin term for optimization, modern research shows improvements in reward modeling and preference optimization.
To detect hallucinations, scientists are discovering ways to define a quantity for "familiarity of context" and detect when and where hallucinations happen by thresholding this quantity. The process includes collecting context vector statistics during training, maintaining an unsupervised model of their distributions, and querying this unsupervised model during testing.
The team has found new ways in R&D for summarization evaluation to calculate the quantity p (original_document | summary) using a strong external LLM that can in turn capture how much information from the original document is retained in the summary automatically for evaluation purposes.
AppTek.ai is a global leader in artificial intelligence (AI) and machine learning (ML) technologies for automatic speech recognition (ASR), neural machine translation (NMT), natural language processing/understanding (NLP/U), large language models (LLMs) and text-to-speech (TTS) technologies. The AppTek platform delivers industry-leading solutions for organizations across a breadth of global markets such as media and entertainment, call centers, government, enterprise business, and more. Built by scientists and research engineers who are recognized among the best in the world, AppTek’s solutions cover a wide array of languages/ dialects, channels, domains and demographics.
