OpenAI’s recent launch of its Decisions API represents a notable advancement in software automation, closely resembling the features of Jev, a model released by TypeSafe AI just weeks ago. At OpenAI’s Dev Day event, CEO Sam Altman explained how the Decisions API allows the Luna model to select from a set of predefined options, such as classifying images or determining agent behaviors.
This API aims to boost speed and efficiency by narrowing the model’s focus to specific choices. Altman stated, “By focusing the model on that choice, we can make it extremely fast while keeping capabilities like image understanding, broad language support, and safety protections.” This strategy mirrors Jev’s operation, which is a super-powered classifier built on a large language model (LLM) and can quickly output probabilities cheaply and at high speeds.
Developers have praised Jev for its rapid decision-making capabilities. Users present Jev with a set of choices, and it promptly generates probability outputs, making it a useful tool across various applications. OpenAI’s Decisions API seems to be the same sort of product, although it is currently only available as a limited preview.
TypeSafe’s CEO, Diogo Almeida, a former OpenAI engineer, commented on the significance of OpenAI’s new API, suggesting it reflects a trend towards “building in a System One compatible way.” This phrase refers to intuitive, quick decision-making, contrasting with the more deliberate reasoning associated with “System 2.” This shift indicates a growing recognition that LLMs as we know them aren’t the right solution for a lot of software because they are comparatively slow and expensive.
Jev has already shown considerable advantages, particularly in monitoring and managing AI agents. After several high-profile incidents involving misbehavior from OpenAI’s agents, the company is exploring cost-effective methods to supervise agent activities. Shapor Naghibzadeh, a long-time cybersecurity professional who leads the start-up QueryStory, believes Jev could play a crucial role in this area. He developed a demo during a recent hackathon that uses Jev to track agent actions. The demo compares each action against the intended task, blocking harmful actions, flagging others for review, and allowing safe actions to proceed.
The cost efficiency of using Jev for monitoring is impressive. While traditional frontier LLM monitoring could cost around $372 per agentic action, Jev achieves similar oversight for just $2.94. This affordability suggests that Jev is arguably cheap enough to run on every agentic action, enhancing the overall reliability of AI systems.
This potential for improved monitoring highlights a broader trend in AI development: the rising importance of decision models. As more companies replicate Jev’s capabilities, the challenge will be ensuring these models produce outputs that accurately reflect real-life scenarios. TypeSafe’s method of generating synthetic data to ensure statistical utility is one approach to maintaining effectiveness in decision-making processes.
With OpenAI’s recent initiatives, it’s evident that the AI decision-making landscape is on the brink of transformation. Integrating models like Jev into agent orchestration systems could streamline operations and enhance the safety and reliability of AI agents across various applications. The implications of these advancements will be crucial as the industry continues to evolve.
