The AI-Based Toxic Content Detection and User Moderation System is designed to automatically detect and manage toxic content in user-generated posts. This system aims to enhance online platforms by leveraging artificial intelligence (AI) for monitoring and moderating posts to improve user experience and prevent harmful behavior. The system is divided into two main components: the Admin Flow and the User Flow. The admin has the ability to monitor user statistics, such as the number of posts, the amount of toxic posts, and account status, while managing registered users. For users, the system allows post creation, including text and images, followed by an AI-driven toxicity detection process through the Google Gemini API. Based on the toxicity levels, users are assigned points, and if the accumulated points exceed a threshold, their account is blocked to prevent further harmful activity. This approach not only ensures real-time monitoring but also promotes a safer online environment. The system is built with HTML, CSS, and JavaScript for the frontend, and Python with Django for the backend. It also integrates Google Gemini API for toxicity analysis. By incorporating automated moderation, the system saves time, reduces human intervention, and maintains a high-quality user experience. This paper discusses the design, development, and implementation of the system, offering a robust solution to detect and prevent toxic content on web platforms.

Keywords: Toxic Content Detection, User Moderation, AI, Google Gemini API, Django, Real-Time Monitoring, Post Statistics.