SHIELD-URL

# Phishing URL Detector

Project Overview

The Phishing URL Detector is a security tool designed to identify malicious phishing URLs, preventing cyberattacks that target individuals and organizations. Phishing attacks commonly deceive users into clicking fraudulent links that mimic legitimate websites, often leading to theft of sensitive information such as passwords, credit card details, and personal data. This project aims to develop an intelligent system that analyzes URLs in real-time, identifies potential phishing attempts, and alerts users about the potential threat.

Objective

The primary objective of the Phishing URL Detector project is to create a reliable, automated solution for detecting phishing URLs before they cause harm. By leveraging machine learning and various URL analysis techniques, the system will be capable of distinguishing between legitimate and phishing websites based on a wide range of features such as URL structure, domain reputation, SSL certificate status, and other behavioral patterns commonly found in phishing sites.

Technologies Used :

- Python: For data preprocessing, feature extraction.

- JavaScript/HTML/CSS: For frontend development, especially if creating a browser extension.

Dataset :

The model will be trained using a comprehensive dataset of phishing URLs, which includes both legitimate and fraudulent URLs. Publicly available datasets such as Phishing Website Dataset from UCI Machine Learning Repository or PhishTank will be used for training the model. Additional data collection through web scraping will be incorporated to keep the model updated.

Expected Outcomes

- A robust URL detector that can accurately identify phishing attempts with high precision.

- Reduction in the number of phishing-related cyber incidents for individuals and organizations.

- Increased user awareness about the risks of phishing attacks and the importance of securing personal data.

1. Imports: It imports necessary libraries like fastapi (for the API), pydantic (for data validation), requests (for making HTTP requests), re (for regular expressions), socket (for network operations), whois (for domain information), urllib.parse (for URL parsing), sqlite3 (for database interaction), logging (for logging events), and others.

2. Logging: Sets up logging to record events (errors, warnings, info) to a file and the console.

3. Database: Initializes a SQLite database to store a list of blacklisted URLs. It creates the table blacklist if it doesn't exist.

4. Data Model: Defines a Pydantic model URLCheckRequest to validate the incoming URL from the frontend. This ensures that the API receives a valid URL.

5. Blacklist Check: Implements the is_blacklisted function to check if the provided URL exists in the blacklist database.

6. URL Analysis: Implements the core logic in the analyze_url function. This function takes a URL as input and performs multiple checks to determine if it is potentially a phishing URL. These checks include:

*Regex Patterns:** Matches the URL against a list of regular expressions known to be associated with phishing URLs. This is a basic form of pattern matching. This is not a very reliable method on its own.

*Hostname Checks:** Checks if the hostname (domain part of the URL) resolves to an IP address (using socket.gethostbyname). If it doesn't resolve, it could indicate a problem. Also checks for unusual characters in the hostname and common misspellings of popular websites.

*WHOIS Information:** Uses the whois library to retrieve information about the domain registration. It checks if the domain is newly registered (phishers often register domains for short periods) or if the domain is registered privately. Be careful with rate limits on WHOIS lookups.

*URL Length:** Checks if the URL is excessively long, which can sometimes be a sign of phishing.

*Suspicious File Extensions:** Checks if the URL path contains suspicious file extensions (like .exe, .scr, etc.), which can be used to trick users into downloading malware.

*HTTPS:** Checks if the URL uses HTTPS. Not using HTTPS is a security risk.

*Unusual Characters:** Checks for unusual characters in the URL.

*Redirects:** Makes a request to the URL and checks if it redirects. Redirects can be used to mask the true destination of a phishing link. This is a basic redirect check. More sophisticated analysis would involve following the redirect chain.

*IP Address in URL:** Checks if the URL contains a raw IP address, which is less common for legitimate sites.

7. API Endpoints: Defines two API endpoints:

* /check_url/: This endpoint receives a URL from the frontend, calls the analyze_url function to check it, and returns a JSON response indicating whether the URL is "safe," "suspicious," or "phishing," along with the reasons for the classification.

* /add_blacklist/: This endpoint allows adding a URL to the blacklist database. This could be used by an administrator or another process.

8. Running the API: Starts the FastAPI server using Uvicorn.

Conclusion

The Phishing URL Detector will be a critical tool in combating the increasing threat of phishing scams. It will provide a proactive defense mechanism against malicious links that could potentially harm users by stealing sensitive information. By integrating machine learning techniques and real-time detection features, this project aims to contribute to the broader cybersecurity field, ensuring safer online interactions for all.