Anthropic Discusses Rules for AI Agent Negotiations

AI company Anthropic is consulting with law firms to address the challenge of how AI agents should negotiate to secure optimal outcomes for human users. These negotiations could involve tasks such as haggling over prices, resolving disputes, or discussing salaries.

Sources indicate Anthropic is seeking legal advice on how lawyers conduct negotiations, aiming for AI agents to represent client interests while ensuring fair results. The company is exploring "negotiation norms" for interactions between intelligent systems, as AI agents are expected to increasingly represent users when dealing with other agents that may have conflicting objectives.

The company is also considering the extent to which AI agents should seek human feedback during negotiations. Additionally, Anthropic is evaluating how cautious agents should be when interacting with other AI systems to avoid excessive suspicion or conspiracy-like decision-making. One proposal involves users signing authorization documents that outline the agent's mandate and governing rules.

Separately, Anthropic has reported shifts in its workforce composition. Over the past year, the company has reduced hiring for junior positions. The proportion of junior employees in its technical departments has fallen from 21% to 14%, with a more significant drop in new junior technical hires from 27% to 9%.