Symflower

We used to and code to 🦾 Here‘s what we experienced:

https://symflower.com/en/company/blog/2024/transpilation-with-llms/

Using LLMs to transpile code (from Go & Java to Ruby) To extend DevQualityEval with Ruby, we used LLMs to transpile Java and Go test repositories to Ruby.

How well do generate code ❓ There’s only one way to find out: models for tasks. Here’s a roundup of popular LLM benchmarks & insights into our take on the topic 🤓

https://symflower.com/en/company/blog/2024/comparing-llm-benchmarks/

Comparing LLM benchmarks for software development A comparison of the most widely used LLM benchmarks for coding, and insights to a new code generation benchmark.

What metrics do you track when evaluating ? 👀 Here‘s an overview of complex statistical and model-based scorers 💡 Bonus: we also cover the that help you get started assessing .

https://symflower.com/en/company/blog/2024/llm-complex-scorers-evaluation-frameworks/

Evaluating LLMs: complex scorers and evaluation frameworks An overview of unsupervised LLM evaluation metrics and the most popular frameworks to help you evaluate models for your use case.