找到 3 篇相关文章
The gossip blogger was on TikTok covered in what appeared to be blood and wielding a knifelike object before the platform cut his stream. He’s since been hospitalized.
Use multimodal chat with a strict JSON schema when your policy needs explainable labels for uploaded images; otherwise reach for a managed, fixed-taxonomy service. There is no dedicated image moderation endpoint here, so the practical design is a policy prompt, a vision-capable chat model, schema validation, and a conservative fallback. That is my short answer. I would not ship the model's prose directly into an allow/block decision. I keep the original decision for audits, translate it into a small internal status, and make the eval set the release gate. The model is one component of the policy system — not the policy system itself. What should a Python image upload moderation example classify for NSFW and violence? The categories should come from the app's actual rules. For a general user-content product, I start with nudity, graphic violence, hate symbols, drugs, and minors-risk. I don't pretend those labels are universal: a medical forum and a marketplace need different thresholds, and a historical archive may legitimately show symbols that a profile-photo product should reject. My first notebook pass is deliberately boring. I assemble a small set of allowed, blocked, and ambiguous pictures; write the expected category labels; and record the policy reason in plain English. Then I run the same prompt and schema across every candidate model. The score I care about first is false negatives on the block set, followed by false positives on harmless uploads. Overall accuracy can hide both. This is also where a JSON schema earns its keep. A response containing "graphic_violence": "high" can be validated, stored, and compared. A paragraph such as “this appears concerning” can't reliably drive a queue or an appeal. Keep the provider response beside a normalized status such as allow , review , or block ; when policy changes, you can replay the raw decisions without migrating every old record. I learned the cost side the annoying way: one evaluation run consumed 18.7 milli
Substack's new Reply Rules feature is currently available for all English-language publications and is designed to give creators greater control over how their audiences respond.