Autonomous AI Newsletter

This document explains the development and deployment of an autonomous AI newsletter system. This system utilizes AI technologies to search for relevant articles from Google News, extract the content using web scraping techniques, summarize the articles using GPT-3 and provides email functionality to distribute the summarized articles to a list of subscribers.

System Components and Functions

Article Search and Retrieval

The system begins by searching for articles. Utilizing the Google News API, the system seeks articles relevant to given keywords.

def fetch_articles(key_word):
    session = requests.Session()
    headers = {
        'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
        'Content-Type': 'application/x-www-form-urlencoded',
        'Connection': 'Keep-Alive',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
    }
    url = f"https://news.google.com/search?q={key_word}&hl=en-US&gl=US&ceid=US%3Aen"
    url_obj = session.get(url, headers=headers)
    bs = BeautifulSoup(url_obj.text, 'html.parser')
    news = []
    for i in bs.find_all('a', class_="VDXfz"):
        try:
            this_news = {}
            url = "https://news.google.com/" + i['href']
            url_obj = session.get(url, headers=headers)
            bs = BeautifulSoup(url_obj.text, "html.parser")
            url = bs.find('a', href=True, rel="nofollow")["href"]
            this_news['url'] = url
            news.append(this_news)
            if len(news) >= 10:
                break
        except:
            pass
    return news

Article Scraping and Summarization

Next, the system scrapes the complete content from each article and employs GPT-3 to generate succinct summaries:

def generate_summary(text):
    prompt = f'''
    your task is to generate a brief summary of the recent news of the
    recent website text, delimited with backticks.
    `
    {text}
    `
    '''
    reply = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful AI assistant."},
            {"role": "user", "content": prompt},
        ]
    )
    return reply["choices"][0]["message"]["content"]

Title Generation

To ensure each summarized article is captivating, the system generates an engaging title:

def generate_title(summary):
    prompt = f'''
    your task is to generate a title for the following article summary,
    delimited with backticks.
    `
    {summary}
    `
    '''

    reply = openai.Completion.create(
        engine="text-davinci-003",
        prompt=prompt,
        max_tokens=15,
        n=1,
        temperature=0.5,
    )

    return reply.choices[0].text.strip()

Email List-serv Integration

Email functionality is integrated into the system to send the curated and summarized articles to a email listserv of subscribers. This feature requires a CSV file of email recipients.

def send_email(subject, body, recipient):
    sender = os.getenv("SENDER_EMAIL")
    password = os.getenv("SENDER_PASSWORD")
    message = MIMEMultipart()
    message["Subject"] = subject
    message["From"] = sender
    message["To"] = recipient
    body_content = MIMEText(body, "plain")
    message.attach(body_content)
    with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
        server.login(sender, password)
        server.sendmail(sender, recipient, message.as_string())

System Operation

The autonomous AI newsletter works in three main steps:

1. Enter the Keyword: The user enters a keyword or topic of interest into the newsletter interface. The system then begins a search on Google News using the provided keyword.

2. Obtain Articles: The system retrieves the top 10 articles related to the entered keyword. It then extracts the full content of each article using web scraping techniques.

3. Send to Listserv: The user can upload a CSV to send the newsletter with the articles, generated summaries and titles, to an email list-serv with a click of a button. Depending on the keyword you search, the article should look something like this.

By following these steps, users can efficiently discover, summarize, and share relevant news articles, providing their subscribers with valuable insights through the autonomous AI newsletter.

Customization

The autonomous AI newsletter system has advanced features which include customization of newsletter aesthetics, application of stylistic writing patterns, and premium features such as automation and source diversity.

Users have the ability to customize the appearance of the newsletter according to their preference. This includes modifying colors (Background Color, Card Color, Text in Card Color, Title Color and Subtitle Color), changing fonts (Title Font, Subtitle Font, Text in Card Font, Title in Text Card Font), and editing components (Change Title, Change Subtitle, Edit Card Content). This provides flexibility in creating a unique user experience for each newsletter.

Stylistic Adaptation

The system offers the capability to mimic various writing styles. Users can choose from a predefined list of styles that include but are not limited to personalities such as Albert Einstein, Elon Musk, and Edgar Allen Poe. This is implemented via a dictionary where each key is the name of the style and the value is a string describing the style, which serves as a prompt for the AI.

style_dictionary = {
    "Albert Einstein": "As one of the greatest physicists in history, Albert Einstein's work revolutionized our understanding of the universe. His theories on relativity and the photoelectric effect laid the foundation for modern physics.",
    "Elon Musk": "Elon Musk, the visionary entrepreneur and engineer, is known for his groundbreaking achievements in the fields of electric vehicles, renewable energy, and space exploration. With his companies Tesla, SpaceX, and SolarCity, Musk aims to revolutionize multiple industries.",
    "Edgar Allen Poe": "Edgar Allen Poe, a master of macabre and mystery, was a renowned American writer and poet. His chilling tales and poems, such as 'The Raven' and 'The Tell-Tale Heart,' continue to captivate readers with their haunting and suspenseful narratives.",
}

Users also have the ability to create their own writing style by providing a custom prompt that is passed in and concatenated to the initial prompt used to summarize articles.

Prompt = "As a comedian, write this newsletter in the style of Kevin Hart, keeping each section to 2 sentences in maximum length. Make it funny.

Premium Features

The system offers premium features to enhance functionality and user experience:

1. Automated Sending: For premium users, the system provides the capability to automate the sending of the newsletters. Instead of manually uploading the CSV file of recipient emails each time, the system will automatically send newsletters to the mailing list every n days.

2. Custom Sender: Premium users can send newsletters from their specific email address, providing a personalized touch to their communications.

3. Extended Data Sources: The system can retrieve and summarize news from additional data sources apart from Google News, providing a more comprehensive view on the chosen topic.

Conclusion

The Autonomous AI Newsletter, armed with cutting-edge AI technologies, promises an efficient and enhanced method of distributing curated news content. It brings together the power of Google News search, advanced web scraping, GPT-based text summarization, and seamless email functionality. With further advancements in the form of customization features and premium functionalities, this system illustrates the transformative potential of AI in redefining how we discover, distill, and disseminate information in a personalized, user-centric manner.