While Silicon Valley argues over whether open source or closed source is safer, a new report has placed open-source AI's dark side on the table. According to The New York Times, entities in China and Iran are using open-source AI models for large-scale influence operations — mass-generating multilingual content, manipulating public opinion and seeking to sway political and social issues in target countries.
Citing researchers and cybersecurity analysts, the report says the core tool of these operations is openly downloadable open-weight large models. Unlike closed models, open weights mean anyone can freely modify, fine-tune or even strip away safety guardrails, making them a low-cost instrument for tasks like opinion manipulation. The entities used open-source models to generate large volumes of seemingly normal posts, comments and news summaries, distributed through social network matrices across multiple language markets. Researchers describe it as a new form of "cognitive warfare" and "information warfare" — no longer hiring human troll farms to flood content, but letting AI produce content at near-zero marginal cost.
Notably, the report distinguishes this from the "malicious agents" long tracked by US security agencies — the latter involves AI actively attacking systems, while this involves AI being used for large-scale content generation and manipulation. This means the "double-edged sword" nature of open models is showing in two dimensions at once: they can be used for defense and empowerment, and they can be cheaply weaponized. For the open-source camp, which has long argued that "openness brings wider participation," the report hands opponents their most needed ammunition; for regulators, it is another example of "open weights enabling malicious use."
Of course, the report acknowledges that the scale and effectiveness of these operations remain under study — the true impact of influence campaigns has never been precisely measurable. But the trend is clear: as open-source models approach closed frontier capability, any country, organization or even individual can obtain a "content-generation weapon" at very low cost. When AI makes manufacturing public opinion as easy as typing, the challenge facing global information space may be deeper than cyberattacks themselves. Where the balance lies between open source and security is a question without simple answers.