Headline Agent: An AI agent that grabs the latest tech news and drops it straight into your inbox

AI agents are autonomous systems that can perform complex tasks on behalf of a user without requiring the user's intervention. It can create its workflows and utilize available tools to complete complex tasks. This system can reason, act, adapt, and make good decisions, which sets it apart from non-agentic chatbots.

These agents ensure the natural language processing functionalities of LLMs, and the features of other tools (i.e, automations, tool calling, etc) to create a 'smart assistant.' So, with this workflow, the agent learns to adapt to the user's expectations over time.

The ability of AI agents to store information about past interactions allows them to prepare for future actions with high accuracy in the comprehension of their responses.

The news headline agent is a practical example of an AI agent; it combines natural language processing, web scraping, and automation to complete a task, bringing results that allow users to get the latest tech news from reputable sources right in their email.

Headline agent is a practical example of how AI agents work, utilizing various tools. Its functionalities come from leveraging LLMs and real-time crawling with Firecrawl to perform certain tasks. The primary purpose of the Headline agent is to grab tech news from reputable websites like Techcabal and other sources across the web and send it to your email in a clean, readable format. This could be a daily email or based on demand.

The internal flow of this agent starts with certain requirements and initialization. We will walk through the overview of how Headline agents work from tool calling to LLMs, and the Email sender stage.

The agent requires a few APIs and tools to perform various tasks in different stages of the process. You can load them in the .env files to securely access to them.

FireCrawl API: This tool is responsible for crawling the web sources and extracting content.

GROQ API runs the LLM via Langchain, which in this case is the Deepseek model.

Tavily Search API is handy for fallback web search, especially when dealing with unknown prompt queries.

GNews API is a backup news source that the agent uses to extract content. Although the primary source is TechCabal, the Headline agent can also leverage other reputable sources like GNews.

SMTP email sender leverages the smtplib to send formatted and structured emails at the end of the whole workflow.

When it comes to tool calling, two primary actions happen with this agent. The headline agent searches for content using the 'get_technology_news' query that is initialized from FireCrawl. This tool scrapes TechCabal, filters it, and returns it to the LLM for processing.

Another tool registered to this agent's workflow is Tavily's advanced search. It helps to retrieve and summarize real-time web results.

LLMs are a big part of AI agents, and this agent leverages Groq's deepseek-r1-distill-llama-70b. This model receives a prompt, checks if it needs a tool, and returns a response. In this sequence, the LLM is instructed on when to use the tool, what tools are available, and how to format the results.

The LLM is tied to a Pydantic schema called 'HeadlineResponse' which has two functions: Ensures that the output contains a list, and makes each list come with a headline and description.

This agent utilizes SMTP email sender; thanks to smtplib, it can parse the final response into headlines, package them, and send them. The smtplib is a Python library that performs this process with a simple function 'send_news_email.'

Now that we've got all the requirements set up, creating a simple UI to wrap this agent is necessary. The obvious option here is Streamlit as it is a clean a fast way to run through the workflow of the the Headline agent.

Let's walk through how to build this headline agent and get it to send news headlines to your email.

!pip install -r requirements.txt

This code installs the required packages for building AI agents and applications using Langchain and LangGraph. These are crucial for integrating models, tools, and APIs. Firecrawl here helps with data web scraping, while 'pydantic' is great for data evaluation. 'python-dotenv' plays the role of managing the environment variables.

The news source for this agent is TechCabal, and it will need to crawl the site to get this news. Tools like FireCrawl and the AI assistant are crucial in this stage of the workflow. The headline agent uses the firecrawl API to get news content from Techcabal's homepage on the date of the request. It can also filter news according to category, as intended in the code line that specifies news that starts with '#startup' and '#fintech.' This returns a response with a detailed headline and description of the news items.

The agent requires sending an email summary using an SMTP Server. It first sets up the sender's credentials and email structure, pulling dynamic news content from news_data and injecting it into a predefined HTML template.

When the agent gets the news content, it formats the email body and creates a loop that structures each news item with a headline and description.

The headline agent can also leverage alternative sources for finding tech news. This code indicates how the agent fetches tech news or performs a web search using AI tools, formats the result, and emails it.

The @tool-decorated search_web function queries the internet using Tavily with advanced depth and limited results. The ChatGroq language model is configured with tools and generates structured headline outputs, which are processed by the call_model function.

The email_sender function takes the final structured message, parses it into headline-description pairs, and sends it via email using a previously defined email function.

The workflow of this agent is quite straightforward. It starts at the LLM node, where it converts the user's input. The LLM then makes a decision based on its input, which triggers the use of various tools available to perform certain tasks (i.e, webscraping or web search). It then proceeds to format and send an email using the email_sender node. Finally, the workflow ends when the email is delivered.

The Link to run the Agent on Google Colab can be found here

Now, we'll create a simple UI to house everything in the headline agent. Let's start by following a few steps, starting from importing the necessary modules to set up the Streamlit app and configuration.

Here are the five steps to run your Streamlit app locally:

Navigate to project directory: cd HEADLINE-AGENT #your project folder path

Install dependencies (Streamlit, python-dotenv, etc.): pip install requirements.txt

Create an .env file for the Environment Variables: TAVILY_API_KEY=your_tavily_api_key, GROQ_KEY=your_groq_api_key, GNEWS_KEY=your_gnews_api_key, FIRE_CRAWL_KEY=your_firecrawl_api_key

Run the app with Streamlit: streamlit run streamlit.py

You can proceed to the app in your browser to see the UI (auto-opens at http://localhost:8501). The UI looks user-friendly and allows you to prompt the Agents, which fetches news updates and then sends them to the email you enter.

So, it's time to wrap this up with a brief guide on how to run the headline agent locally:

The first command requires you to install the requirements.txt file: pip install -r requirements.txt

Finally, we'll use Python 3 to run the code and execute every line of code in it: python3 agent.py

And your output looks as in the image below;

The Headline agent is a practical demonstration of how AI agents easily integrate various tasks and automated workflows. This agent uses web scraping, LLM reasoning, and agentic workflows to send tech updates to your email. It ensures that effective tools like FireCrawl, Tavily Search, and Pydantic are used to provide real-time updates.

Headline agents only demonstrate a glimpse of how you can use Agentic workflows to automate your work, and there are lots of capabilities these agents have yet to explore.

The full source code can be found here, feel free to star and fork the project, happy coding!

A big shout-out to Maigari David for writing the article!