Article reprint source: AIcore
Original source: AIGC Open Community
Image source: Generated by Unbounded AI
Last Tuesday, the AIGC Open Community introduced Writer, a generative AI platform that raised $100 million. The company was able to obtain a total financing of $126 million in just three years and become one of ChatGPT's main competitors, which is inseparable from its superb technology. At the same time, it fully proves that its model has successful application cases and has been recognized by capital and users.
Currently, Writer has open-sourced its large language model Palmyra on huggingface. There are 8 models, including small, base, 20b-chat, Instruct-20b, med-20b, etc., which can be used commercially and support data fine-tuning.
Open source address: https://huggingface.co/Writer
Free online trial address: https://app.writer.com/organization/
Palmyra's technical highlights include: small parameters and powerful functions, which are very helpful for small and medium-sized enterprises and individual developers without computing resources; it has received business writing and marketing data training, mainly for corporate users; enterprise-level data security, with multiple built-in security guardrails;
In addition to generating text, it can also extract content summaries of videos, PDFs, and audios; it supports data fine-tuning, and companies can create their own "ChatGPT" assistant.
Below, the "AIGC Open Community" introduces several special models of Palmyra:
InstructPalmyra-20b
This is an instruction tuning model built on the Palmyra-20b base model, supporting advanced natural language processing and tailored needs.
The InstructPalmyra-20b model was carefully trained on an extensive dataset of approximately 70,000 command-response records generated by Writer’s expert language modeling and fine-tuning team.
InstructPalmyra-20b has an excellent ability to process complex instructions and generate accurate and contextual responses. This makes it an ideal model for developing a wide range of applications such as virtual assistants, customer support, content generation, etc.
Furthermore, the model’s comprehensive training enables it to adapt and perform well under different conditions and contexts, further expanding its potential application cases.
Palmyra-with-20b
Palmyra-Med is a model that Writer built specifically for the needs of the healthcare industry, with instructions fine-tuned based on medical data.
Palmyra-Med achieved the highest score in a test on PubMedQA, a leading biomedical question answerer, with an accuracy of 81.1%, outperforming both GPT-4 and medically trained human testers.
It can provide functions such as translating professional medical terms, extracting medical notes summaries, analyzing massive medical data, and automatically generating medical insights.
Palmyra Large 20B
Palmyra-Large is a causal decoder model built by Writer, which is enhanced by Palmyra-Index-Data and trained on 800 billion data in a high-quality corpus.
Palmyra Large uses the Causal Language Modeling (CLM) objective during model pre-training. Similar to GPT-3, it is pre-trained using the self-supervised causal language modeling objective.
The model runs very efficiently and consumes very little resources. It is suitable for business scenarios such as healthcare, marketing, market, IT, design, human resources, etc. to create a customized AI assistant.
Performance Evaluation
Palmyra achieved the highest score on Stanford HELM, surpassing well-known open source models such as Falcon 40B and LLaMA-30B. HELM is a well-known benchmark platform of the Center for Fundamental Model Research at Stanford University.
Palmyra ranked first in several important tests, scoring 60.9% on Large-Scale Multi-Task Language Understanding (MMLU), 89.6% on BoolQ, and 79.0% on NaturalQuestions.
Palmyra ranked second in two other key tests, with a contextual question answering score of 49.7% and a TruthfulQA score of 61.6%, showing very strong overall performance.
In short, Palmyra is very worthy of developers who want to commercialize large language models to study its model architecture and functions and learn from its successful experience.