The training is split between general AI training sessions and project-specific courses.
During general training, we ensure an expert understands the main concepts of what it takes to train a model or an agent. They learn about pre- and post-training, how to craft correct justifications, and why it’s important to follow instructions. They have to pass a soft exam to verify their understanding.
After that, they can select a project which we think they might be a fit for. Within the project, they undergo two additional training sessions. One is designed around the category of a project — SFT, Evals, etc. — and the other one is focused only on the project itself. Here experts learn about rubrics and why they’re designed the way they are; we go deeper on justification and how to make it the most useful. They have to pass the exam and do a few shots to prove their ability to do a quality job.
Then, they go into project-specific training. Here they learn about the platform they’ll be working on (if staffed outside of the Lumos platform). We go into details of the project’s instructions, spending the majority of the time covering good and bad examples. Then they pass one more exam and a benchmark wall to yet again confirm they understood what to do, as well as confirm their domain skills.
And so they can begin the project.