> ## Content Index > Fetch the complete content index at: https://www.tao.media/llms.txt > Use this file to discover other available public pages before exploring further. # OpenAI Releases MultiLangBench as GPT-6 Models Improve COBOL and Fortran Coding - URL: https://www.tao.media/openai-releases-multilangbench-as-gpt-6-models-improve-cobol-and-fortran-coding/ - Published: 2026-10-20T20:45:13.000Z - Updated: 2026-10-20T20:45:14.000Z - Description: New evaluations test whether AI can build database software in less-represented languages and migrate COBOL without changing its behavior. - Author: Tristan Hillerich - Tags: OpenAI, AI, News OpenAI has [released MultiLangBench](https://openai.com/index/low-resource-programming-languages/?ref=tao.media), an evaluation for building database software across 20 programming languages, alongside results showing improved performance by its GPT-6 models in languages such as COBOL, Fortran, OCaml, and Visual Basic. GPT-6 Astra achieved an average test pass rate of 87.4%, compared with 26.8% for GPT-5.6 Sol. In a separate evaluation of COBOL migration, GPT-6.1 Sol reached 97.9% accuracy at higher reasoning effort. That test measures whether a model can rewrite existing programs in another language while preserving their observable outputs exactly. The results address a practical problem for teams maintaining older software, where coding assistance needs to work beyond languages with abundant public examples. OpenAI said [developer and enterprise feedback](https://x.com/OpenAI/status/2112310345863930168?ref=tao.media) helped guide the work toward systems whose business rules may survive primarily in code, with incomplete documentation or original developers no longer available. ## Building a Database Across 20 Languages MultiLangBench requires a model to implement a SQLite-compatible database engine from scratch in a specified language. Rather than solving isolated programming exercises, the model must organize a codebase, manage state, handle errors, and extend functionality without breaking features it has already built. The evaluation uses database files as a common interface across languages. Each candidate implementation receives an initial database and a SQL query, then produces a modified database. The reference SQLite engine checks that resulting file to determine whether its contents match expectations. This allows implementations written in different languages to face the same functional checks. OpenAI reports the following test pass rates for four of the less-represented languages, with GPT-6 Astra evaluated at maximum reasoning effort: | Language | GPT-5.6 Sol | GPT-6 Astra | | ------------ | ----------- | ----------- | | COBOL | 10.9% | 72.1% | | OCaml | 25.6% | 85.8% | | Fortran | 21.4% | 83.5% | | Visual Basic | 27.5% | 89.4% | Astra improved in every language tested and scored above 83% in 19 of the 20 languages. COBOL remained the hardest, at 72.1%. The language set also includes widely used options such as Python, JavaScript, Java, C++, and Rust, allowing the same task to measure performance across different programming environments. The [MultiLangBench repository](https://github.com/openai/MultiLangBench?ref=tao.media), released under Apache 2.0, provides the setup and test suite. Its instructions target SQLite 3.40.1 and prohibit internet access during evaluation. Existing SQLite implementations must be removed from the model's environment so the candidate cannot delegate the work to an installed database engine. The repository contains roughly 24,000 tests converted from SQLite's approximately 51,000 TCL tests. For routine evaluation, it recommends a 272-test subset to reduce execution cost, with the full suite available for more precise measurement of high-scoring implementations. A solution must use the requested programming language; implementations using another language or a mixture of languages fail the language requirement. ## Preserving COBOL's Financial Logic The migration evaluation tests a different engineering skill, changing a program's implementation without changing what it does. Models translate COBOL programs into C++, Java, Kotlin, Python, or Rust, then face hidden tests requiring byte-for-byte parity on observable outputs. A migration counts as successful only when it passes every test. At low reasoning effort, GPT-6.1 Sol achieved 86.6% accuracy, up from 5.4% for GPT-5.6 Sol at the same setting. Increasing reasoning effort brought the newer model to 97.9%. Preserving behavior requires more than translating syntax. COBOL's fixed-point arithmetic, for example, discards excess decimal digits unless rounding is explicitly requested. A rewrite that rounds instead may compile and run normally while producing a different financial result. Missing-record handling can similarly encode business requirements that a replacement program must retain. Those details explain why an all-tests-pass migration score differs from MultiLangBench's test pass rate. The database evaluation measures the proportion of working functionality across tests; the migration evaluation requires an entire program's tested outputs to match. The two percentages describe different units of success. For production migrations, OpenAI says expert review and testing against a system's business rules and dependencies remain part of the process. That work checks the behavior an organization actually relies on, beyond the programs and hidden cases in the evaluation. ## Lower-Cost Coding Options OpenAI also reported gains from GPT-6 Luna, which achieved a 40.3% average MultiLangBench test pass rate at maximum reasoning effort. Its estimated average API cost was $0.77 per sample, compared with $3.48 for GPT-5.6 Sol, a reduction of approximately 78%. Luna improved over that baseline in every language tested, including COBOL at 17.1%, Fortran at 31.4%, OCaml at 34.1%, and Visual Basic at 41.6%. It offers a different cost-performance tradeoff from Astra's higher test pass rates. In the COBOL migration evaluation, GPT-6.1 Sol's low-reasoning result cost 61% as much as GPT-5.6 Sol's, or 39% less. These are costs for the evaluated workloads, rather than a change to API list pricing. The models are available through the API. Developers can use the released MultiLangBench tests to inspect database functionality in their own programming languages and compare both test coverage and API spending.