{Article: Model Context Protocol}

(Source)

So there’s been a lot of noise about MCP. I saw many claims that it was basically AGI and completely useless. So I decided to read up on it straight from the source. No explainers, no blogs, no videos.

Here are my notes:

(If you want just a high-level overview, I made a diagram)

{Code}

I also wrote a minimal implementation of an MCP server here:

https://github.com/Prathmesh2498/MCP-Practice

• The most useful thing to separate fluff from the real stuff is the definition given on the first page, first para:

  MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.

• The most important part to focus on in MCP is P. It’s a protocol — a standardized way to feed data into an LLM.

• The easiest use case of this is if your LLM is independent of your data ingester, you can plug and play LLMs into your app and see which one works best. What enables this? — standardization, i.e. protocol.

• They mention 3 core components that make up MCP:

  • Protocol Layer: The protocol layer handles message framing, request/response linking, and high-level communication patterns.

  • Transport Layer: Either Stdio or HTTP over SSE

    (Server-Sent Events (SSE) is a method for pushing data from a server to a client over an HTTP connection, enabling real-time updates without the need for constant polling.)

  • Messages: Can be Requests, Results, Errors or One-way Notifications.
    One-way here means a notification that expects no response, can be sent from either the client or the server.

• The Client-Server will use these to initiate a 3-way handshake:

• Resources are a core primitive in the Model Context Protocol (MCP) that allow servers to expose data and content that can be read by clients and used as context for LLM interactions — like files, DB data, API responses, media, etc.

• Prompts enable servers to define reusable prompt templates and workflows that clients can easily surface to users and LLMs. They provide a powerful way to standardize and share common LLM interactions. — Basically things we want to do over and over will be done via predefined prompts, it has a well-defined structure, and you can discover them from the prompt/list endpoint.
  For example:

  {
    name: Reality Check;
    description: Forgot to code without LLMs; 
    arguments?: [              
      {
        name: Error;
        description?: Just paste the error dummy;
        required?: True
      }
    ]
  }

• The response will contain the well-defined prompt with the error that you feed to the LLM.
  

• Tools are functions -_-

  @app.list_tools()
  async def list_tools() -> list[types.Tool]:
      return [
          types.Tool(
              name="calculate_sum",
              description="Add two numbers together",
              inputSchema={
                  "type": "object",
                  "properties": {
                      "a": {"type": "number"},
                      "b": {"type": "number"}
                  },
                  "required": ["a", "b"]
              }
          )
      ]
  
  @app.call_tool()
  async def call_tool(
      name: str,
      arguments: dict
  ) -> list[types.TextContent | types.ImageContent | types.EmbeddedResource]:
      if name == "calculate_sum":
          a = arguments["a"]
          b = arguments["b"]
          result = a + b
          return [types.TextContent(type="text", text=str(result))]
      raise ValueError(f"Tool not found: {name}")

• Sampling allows the server to ask completions from the LLM.
  I did not understand the use case of this, but here’s something I found on a DigitalOcean Blog.
  […] while clients maintain the authority to decline potentially malicious requests or limit resource usage. This approach is valuable in scenarios where clients interact with unfamiliar servers that still require access to intelligent capabilities. (Source)
  If you know of any specific use cases here, let me know!

• Roots are soft boundaries (defined as URIs) which the client can send to the server. One thing that seems useful is prioritizing operations for resources within roots compared those outside the roots.

{Return}

I hope you enjoyed this newsletter. Let me know if you have any feedback.

Next week पुन्हा भेटू!

(Source)

A good primer on what passkeys are and how they are integrated by Grab in the auth flow. One thing I did notice was that passkeys are supposed to make user login easier by not making users remember passwords. However, the private keys are stored on the device (Icloud for Apple). This takes away the control from the user. If my private keys are managed by an external service, I become dependent on that service.
Still, I feel it’s a good read nonetheless.

{Article: Kafka Internals}

(Source)

A detailed post that walks through the Kafka source code. It shows how Kafka handles new zookeeper vs Raft, Socket connections, and writes logs.

{Paper: Textshop - Photoshop For Text}

(Source)

A new tool which reimagines image operations on text.

In addition to dragging to change sentence ordering, resizing to expand or summarize, and rotation for changing word orders, it also introduces tools like:

• Brush – Adjusts the tone of selected text based on the chosen style.

• Tone Eyedropper – Captures the tone of selected text and updates the tone picker.

• Smudge – Rewrites selected text for variation or smoother transitions.

• Singularizer and Pluralizer – Changes number agreements in text.

• Tense Changer – Converts text to past, present, or future tense while maintaining consistency.

• Repair – Fixes grammar, punctuation, and spelling errors.

• Eraser – Deletes text while keeping sentences grammatically correct.

• Prompt – Applies a user-defined prompt to modify selected text using AI.

{Return}

I hope you enjoyed this newsletter. Let me know if you have any feedback.

Next week पुन्हा भेटू!

{System Design: Managing 1.5 Trillion Kakfa Events}

(Source)

This is a great read on how Agoda managed 1.5 trillion Kafka events daily. The key insight was to separate the concerns of generating events and sending them to Kafka clusters. The dev team wrote to a cold data store, and then another dedicated service managed using the data with required metadata and wrote to the Kakka clusters.
What I like about this blog is that they have also explained the tradeoff considerations, which I feel is a critical part of any system design.

{Article: Pytorch Internals}

(Source)

This article dives deep into how Pytorch is implemented, including Tensor structures, strides, index calculation, operator operations, how to make tensor extensions, how auto-grad works, etc.

{On Acing Your Remote Interviews}

(Source)

This is a surprisingly tempered and practical guide for things to consider when appearing for remote job interviews.

{Return}

I hope you enjoyed this newsletter. Let me know if you have any feedback.

Next week पुन्हा भेटू!

Let me know if you like this format.

Here we go!

Week 1: 20th March - 26th March

{Blog: Why len(list) == 0 is slower than not list}

(Read Here)

I read this interesting blog post that shows using len(list) == 0 is 2x slower than using not list.

Well turns out the reason is because of the len() function call overhead.

The post is paywalled, but I used the dis package to generate the bytecode and it shows the overhead for the len(list) function call. The code on the right calls the len(list) function.

The layout of CPython is explained really well in the article.

{Paper: Dynamic Tanh}

(Read Here)

This paper is very popular right now since Kaiming He and Yann Lecun are among the authors and it proposes a very simple but potentially ground-breaking idea.
The paper proposes to replace Layer-Norm with a parameterised Tanh function.

The key observation was that if you consider all data points together, Layer Norm’s linear transformation for each point actually looks like a scaled tanh since smaller inputs have larger slopes and vice-versa.

This image from the paper shows this effect:

They claim to have equivalent or better accuracies on different models like ViT and ConvNeXt as well as LLaMA models. In addition to this, they also claim an 8% reduction in inference time for LLaMA 7B. It is a good read!

{System Design: How GitHub Rebuilt Their Push Processing System}

(Read Here)

I want to consistently read about system design so I will try to include at least one article every week. This one talks about how GitHub used Kafka to decouple their push processing system. And the advantages and disadvantages of a decoupled system.

{Return}

I hope you enjoyed this newsletter. Let me know if you have any feedback.

Next week पुन्हा भेटू!

So I am working through Andrej Karpathy’s makemore series to understand better how different ML models work. In his bi-gram model video, he talks about the keepdims parameter in the tensor.sum function and how understanding it is important. So I thought I’d write a post (mostly for myself), so I can come back and revise what I have learnt.

Let’s start with the basics:

The 2 conditions for tensors to be broadcastable is

The official documentation has many examples, but I will just use the example from the tutorial.

In the bi-gram model, the first step is to create a n*n character array that stores the frequency of each pair or bi-grams. For example row[1][1] stores how frequently the bi-gram ‘aa’ appeared in the dataset. Additionally, we also add a ‘.’ symbol to denote the start and end of every sequence hence we get a 27*27 array, with row 0 being the special symbol ‘.’ and rows 1 to 27 being characters a-z. Any position [i][j] holds how many times character j followed character i.

Now the way we build the model is to sample from a multinomial probability distribution of these counts. To do that we first need to convert the raw counts to probabilities that sum up to 1. To do this, we use the following code:

The generator is just to get consistent results.
P is a tensor that stores the counts (stored in N as ints) as floats.
Then we sum along the columns and set keepdim to be True.

The result is a tensor of size [27, 1]

Now according to the rules, tensors of size 27*27 and 27*1 are broadcastable if from the right, each pair is either 1, equal or non existent.

Pair 1, 27 → One of them is 1

Pair 27, 27 → Equal

Internally, Pytorch copies the column vector 27 times and does element-by-element division.

Now to the problem,

If keepdims is false, we get a tensor of size [27]. If we check [27, 27] and [27, ] are broadcastable, they are since in the first pair, one of dims is non-existent and hence allowed. So the code runs without errors, however, the output produced is complete garbage. See the example below:

These are supposed to be names! Granted that the bi-gram model is pretty poor at estimating names, but the first image has at least lengths that somewhat resemble names with keepdims = False, it is producing sentences!

So what is happening here?

Look at the image above, in the first case, we have the shape (27,1) hence we sum across columns and then get a column vector which is copied 27 times across. However, when we implicitly get (1,27) due to missing dimensions, we sum across rows and get a row vector which is copied horizontally 27 times.

Hence in case 1, we correctly divide the count of say ‘.b’ with the sum of all instances where the the prefix is ‘.’ , But in the second case, we divide with instances where b followed some letter, which is incorrect!

This can be a tricky bug given that neural nets and ml models can produce incorrect results for myriad of reasons and hence when you have a complex model, it’s easy to miss this simple reason.

I would love to read any more tutorials you might come across, feel free to send them to me!

I hope you found this article useful. Until next time!

📖Resources

1. Official Documentation - Broadcasting Semantics

2. Bigram Video

Let’s connect :)

Twitter | Instagram

Before we start with this post, go here if you want to know why I was inactive for the last few months, I am back to posting regularly now :)

This is an interesting paper I read recently that has a novel way to parse user questions about a document. The key insight is that articles or research papers have a very structured format. In some cases, strict templates need to be followed. When this is true, it makes little sense to get context from just blobs of text which is what current approaches follow. PDFTriage proposes a way to train the model not only on the content but also on the metadata which provides the structure of the document. This is helpful because the AI model can now understand the mental model the user had in mind when writing the article.

This enables PDFTriage to accurately answer queries such as “Summarize pages 3-7”, “analyze data in table 3”, etc. In this post, let’s take a look at how it works.

Why do we need to have pre-processing?

It’s trivial but important to note that we need the pre-processing stage since for long documents it’s computationally infeasible to have the entire context in memory. And when we are analyzing tomes — which is what we ideally want our NLP Agents to do — we need to only extract the information relevant to the user query to get useful results.

Architecture

PDFTraige has 3 stages, Getting the document structure, Using the structure to get the relevant context from LLMs and then using the context to answer user queries.

This is the figure presented in the paper:

Stage 1: Generate Document Structure

The authors consider born-digital PDF documents — documents that are created completely electronically (not scanned) — and use the AdobeExtract API to create a DOM-like structure of the document. This is then converted to a JSON file and given as input to the LLM.

Stage 2: LLM Querying

As mentioned above, the input to the LLM is the JSON object which represents the structure and the following 5 functions:

• fetch_pages

• fetch_sections

• fetch_table

• fetch_figures

• retrieve

The paper just mentions “each function allows the PDFTriage system to gather precise information related to the given PDF document, centring around structured textual data in headers, subheaders, figures, tables, and section paragraphs.”

Each function is used in a separate query to get multiple answers and combined to get the final context.

A small note on how to call the functions

OpenAPI has a detailed example here. However, I’ll just briefly explain how a function could be called based on the query.

You first define the function in Python, then pass it as a JSON object which has keys like description, parameters, etc.

This is then passed as an argument to the OpenAPI model call. In the response, the model returns an array of the function calls it needs to make based on the user query. You use this array to call the functions and then run a second query where you input the original query and the response of the function calls requested by the model.

Stage 3: User QnA

This is the prompt used to initialize the system:

“You are an expert document question-answering system. You answer questions by finding relevant content in the document and answering questions based on that content. Document: <textual metadata of document>”

The model uses the functions described above in addition to the context retrieved in stage 2 to output the final answer.

Conclusion

Even though the approach seems simple enough, I enjoyed reading this paper because it shows how we can make improvements to AI models by literally thinking about what a human would do in the given context. Since we have a mental model when writing anything, like this post for example, it makes sense that the AI model should be able to parse and reference it to get better answers.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. PDFTriage Paper

Let’s connect :)

Twitter | Instagram

CM3Leon (pronounced chameleon) is a retrieval-augmented, token-based, decoder-only multi-modal language that can not only generate images but also infill images as well as generate and infill text. It combines image and text generation using a combination of models to achieve performance comparable to diffusion models.

Let’s take a look at how it works!

Design

The pre-training stage has two steps

Image Tokenization

The image tokenizer tokenized a 256x256 image into 1024 tokens from a vocabulary of 8192 tokens. Here, vocabulary means the set of all possible tokens. For the text, the process uses a pre-trained transformer with a vocabulary of 56320. In addition to image and text, is also a special token <break> which is used to differentiate between them.

Retrieval Augmentation

The retriever takes a query q, and a candidate document m, and returns a relevance score r(q, m). Using a CLIP-based architecture, the text and image parts are separated and tokenized. Then the average of the two vector representations is used as the vector representation of the document.

Training

While training, 3 parameters are used for selecting the candidate documents:

1. The document selected should be relevant to the input sequence x.

2. The document should contain images and text instead of just one of the two.

3. While selecting top K documents, no two documents should be too similar.

Objective function

The objective function of CM3Leon is the next token loss -log p(x) where x is the input sequence.

Here’s an example of how all of it works together

Conclusion

CM3Leon is in line with the last few posts which combine LLMs and vision transformers to get good results in not only image generation but also image description. This will be the last post where I just break down the paper on a general level, from the next post, I plan to explore classic research papers including all the math as well as code if possible. I am excited about what’s next! Let me know if you have any suggestions on which topics I should cover.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. CM3Leon Paper

Let’s connect :)

Twitter | Instagram

LENS (Large Language Models ENnhanced to See) is a model that combines the reasoning capabilities of LLMs with computer vision. It combines vision-based models to get general-purpose data on the image and then uses an LLM with context to answer user-defined queries.

In this post, let’s take a look at how it works.

Working

LENS consists of 3 different vision modules and 1 language module. The vision modules generate all information irrespective of the user query. Then, this information and user-specific context are given to the LLM to generate the output.

Here’s what each model is responsible for:

Tag Module

The module generates the most appropriate tag for a given image using the CLIP vision encoder. The prompt is “The photo is of _______”

Attribute Model

The attribute model uses a pre-trained CLIP model to generate descriptions of the image.

Intensive captioner model

It uses the BLIP model to generate N captions that capture diverse aspects of the image.

Visual Vocabularies

These act as a connecting dot between the image and the text. The tags are obtained from various datasets such as object detection and semantic segmentation, and genome. Attributes data uses GPT-3 to get descriptions for various object types. These are then used by the vision modules to generate a description of the image free of user query context.

Reasoning Module

This is a trained LLM module that gets as input the output of the 3 vision modules as well as the use query.

Here are some of the examples given in the paper

Conclusion

LENS is an example of adapting LLMs to vision tasks. This will make to make LLMs more general-purpose and work with images and eventually with videos and other forms of visual input as well.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. LENS Paper

Let’s connect :)

Twitter | Instagram

Subscribe now

The result, is phi-1, a model that performs better than traditionally trained code generation LLMs.

Let’s take a look at how it works.

The problem with training on online code

There are a few problems when you train the LLM on online codes found on Stackoverflow, CodeContest, etc. They are:

1. The codes are not often self-sustained i.e. external information is required for the code to make sense.

2. A lot of boilerplate code gets repeated.

3. Examples might not have any algorithmic value, i.e. a lot of code that is just manipulating data and passing it on without actually doing any meaningful contributions.

Working of phi-1

Phi-1 uses 3 datasets:

1. Filtered-code language dataset

2. Synthetic Textbook

3. Synthetic Exercises

The filtered code is obtained by prompting GPT-4 to filter based on educational quality to a student whose objective is to learn code.

Here’s an example of a positive and negative sample:

The synthetic textbook is generated using prompts that mention different levels of understanding to generate diversity.

The exercises are generated by giving a doc string of the code that needs to be completed.

The training is done first on the CodeTextBook (Filtered code-language + Synthetic Textbook) to get phi-1-base. The base model is then fine-tuned on the exercises to get phi-1.

Conclusion

The authors show that phi-1 performs well on various tasks. The reason I choose to write about this paper is it shows how using small tweaks in data used for training as well as fine-tuning, superior performance can be achieved.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. Textbooks Are All You Need

1. Attention is all you need (Other than the title inspiration, this has nothing to do with this paper. But if you haven’t read this paper, you must!)

Let’s connect :)

Twitter | Instagram

In this post, we take a look at another model that works on a similar principle of using a sophisticated Chain-Of-Thought (COT) prompting approach to plan and execute tasks over long documents.

The model proposes 3 steps: Action Mining, Plan Generation and Plan Execution.

Let’s explore how it works.

Model Design

Action Mining

The model generates all possible steps that might be required for the task. The output of this step is a function definition and it’s a description in natural language.

The module gives a few seed actions along with the task to get task-specific results.

As seen above, the output is a function definition with parameters and a function description in natural language.

A full pass over the training data is required to generate the set of all actions.

Plan Generation

This is the framework for producing the answer. It will choose the actions mined in the previous step. The input arguments to action could be the input document, a string or output from the previous step.

Plan execution

The actions from step 2 are executed in this step. The goal is to execute actions in the right order as well as handle errors where necessary.

A plan parser is used prior to plan execution for error handling and checking the quality and feasibility of the generated plan. In case of an error, the error message is fed back to the LLM and a new plan is generated.

Limitations

Even though this is a novel approach to using LLMs of generating misinformation or hallucination. There were some cases in the evaluation where the plan generated was incorrect or a good plan was executed in an erroneous way. In addition to this, even though the model is designed to work on large documents, it is bottlenecked by the max context size of the LLM.

Conclusion

Auto planners using LLMs are in the nascent stage of development. As LLMs get more powerful, better architecture can be designed to build an efficient planner of difficult tasks. This will have huge applications in education since once the model is tuned to a particular student, a smart planner can plan an action map suited to each individual student.

I’m sure we are in for exciting times ahead!

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. PEARL Paper

Let’s connect :)

Twitter | Instagram

In line with these advances, ViperGPT is a new LLM based model that uses LLMs as well as some vision models such as GLIP, MiDaS, X-VLM in conjunction with Codex to create a model capable of answering questions based on images and videos.

The use of LLMs to generate code and pre-trained models to make the code generation fast and efficient makes this model interesting.

Let’s take a look at how ViperGPT works.

Design

The input to the program is a visual input (image/video) and a user query related to the visual input.

The first step is to generate a program using the query q,

z = ⫪(q), where ⫪ is the program generator

Then using the program z, and the visual input v, we get the result r.

r = Ø(x, v), where Ø is the execution engine.

The program z is generated in Python using OpenAI Codex API to allow flexibility as well as programming tools such as loops and conditional statements.

Implementation

Not that we’ve seen the high level design, let’s look at how each of the components work, starting with the program generator ⫪.

Program Generator ⫪

In previous approaches, ⫪ was generated using a neural network. However this has certain drawbacks such as

1. At scale, supervised learning is not efficient due lack of available data.

2. The computation costs also grow with size.

ViperGPT uses LLMs to autoregressively* generate code based on the user query. Since the optimization is done using LLMs, all the python code available on the internet can be used for fine-tuning, hence removing the issue of scale in traditional approaches.

*Autoregressive prediction - Predict token at time T based on a few token x before time T i.e. for computing T use tokens T-1, T-2, ….., T-x.

The input to the LLM is the API specification of certain pre-trained models along with the user-query. Since the exact implementation is not provided it allows for greater flexibility in the output.

For example:

def find(self, object_name: str) -> List[ImagePatch]:
 """Returns a list of ImagePatch objects matching object_name contained in the crop if any are found.
		Otherwise, returns an empty list.
		Parameters
		----------
		object_name : str
		the name of the object to be found

		Returns
		-------
		List[ImagePatch]
		a list of ImagePatch objects matching object_name contained in the crop

		Examples
		--------
		>>> # return the children
		>>> def execute_command(image) -> List[ImagePatch]:
		>>> image_patch = ImagePatch(image)
		>>> children = image_patch.find("child")
		>>> return children
		"""

Example API Spec input to the LLM

Let’s look at an example from the paper for the task of Visual Grounding, which means to find out spatial relationships and visual attributes of an image. Following APIs of the pre-trained model are given as input to ViperGPT: (model names are in parenthesis)

1. find (GLIP): takes as input an image and a short noun phrase (e.g. “car” or “golden retriever”), and returns a list of image patches containing the noun phrase.

2. exists (GLIP): takes as input an image and a short noun phrase and returns a boolean indicating whether an instance of that noun phrase is present in the image.

3. verify_property (XVLM): takes as input an image, a noun phase representing an object, and an attribute representing a property of that object; it returns a boolean indicating whether the property is present in the image.

4. best_image_match (X-VLM) takes as input a list of image patches and a short noun phrase, and returns the image patch that best matches the noun phrase

5. best_text_match(X-VLM): takes as input a list of noun phrases and one image, and returns the noun phrase that best matches the image.

6. compute_depth (MiDaS) computes the median depth of the image patch.

Using these methods of pre-trained models as well as the user query, the code z can be generated. Now let’s look at the execution engine Ø

Execution Engine Ø

At execution time, the generated program z accepts an image or video as input and outputs a result r corresponding to the query provided to the LLM.

The execution engine uses a Python interpreter to run the code generated by the program generator. This allows it to use built-in functions and tools such as sort, if/else, loops, etc.

You can see the full code here, but here is how the code is executed using python’s exec() function:

"""
exec is used to interpret a string as python code
"""
exec(compile(code_line, 'Codex', 'exec'), globals())
result = execute_command(im, my_fig, time_wait_between_lines, syntax)  # The code is created in the exec()

Examples:

Here are a couple example of ViperGPT in action:

LLMs are helping to build tools that help to give better explanations about raw data. ViperGPT is one of the first tools that can generate python code to preprocess data and use the results in conjunction with LLMs to add better context to the code execution results. I sure had fun reading this paper.

Let me know your thoughts.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. ViperGPT Paper

2. GitHub Repo

3. IMG1 was made using Vector stock images, below are the attributions as per free license:

   1. https://www.vectorstock.com/royalty-free-vector/random-icons-set-vector-7991904

   2. https://www.vectorstock.com/royalty-free-vector/25-engine-icons-vector-36780597

   3. https://www.vectorstock.com/royalty-free-vector/site-icons-vector-45958151

   4. href="https://www.vectorstock.com/royalty-free-vector/form-icons-vector-45958593"

Let’s connect :)

Twitter | Instagram

However, as the size of the LLMs grows, the LLM tends to “forget” some things resulting in poor results. Hence, the authors propose an internal memory module to improve LLMs' generalizations and adaptability.

Let’s take a look at how DT-Mem works.

Model Setup

The problem is formulated as a Markov Decision Process (MDP). This means for every state transition Sn → Sn+1, the inputs as well as some x previous states are taken into consideration.

The MDP tuple consists of the following elements:

S - Set of states

A - Set of Actions

R - Reward Function

p - Transition Kernel (More on this below)

γ - Discount Factor

The transition kernel p is defined as S1 * A * S2 → (0, 1) which can be thought of as the probability of reaching state S2 from S1 by taking action A.

LoRA

A second essential component is LoRA or the Low-Rank Adaption method. This is used to map old knowledge to new tasks. The idea behind LoRA is to learn a low-rank matrix for a new task and then use it to get the relevant subspace from the old task.

An intuitive way to think about this is using the vector space of the old tasks, which is large, and extracting only the parts relevant to the current tasks from that space.

Architecture

DT-Mem consists of three components: the Transformer module, the Memory module, and the Multi-layer perceptron (MLP) module.

Transformer Module

It takes in a sequence of n tokens and gives the output as one of attended state embeddings, action embeddings, or return-to-go embeddings.

Working Memory Module

This module is based on the adage “Think before you act” and works by identifying the salient information, deciding where to store the information and how to integrate the new memory with the current memory state.

The memory is initialized as a random M*N matrix where each location (i, j) represents a slot.

Then using content-based addressing that uses the attention mechanism, the right slot to store the memory is identified. The intuition behind this is that we store similar information together.

To merge the new memory with the current state, two new vectors are used, the erasing vector and the addition vector. The erasing vector selectively removes information from the memory matrix and the addition vector selectively adds new information.

To retrieve the information, the same attention mechanism is used to identify the slot most relevant to the query and then read from that slot.

MLP

The multi-layer perception module is similar to the GPT-2 model, only it is decoupled from the transformer module and has the memory model preceding it. It produces the final result by taking the combined embedding from the transformer and memory modules.

Conclusion

This model was able to work well in Atari and Meta-world environments. Using just the weights of the neural network poses challenges that are solved by having a memory module.

Such new and innovative ideas are coming up every week. It feels like an inflexion point and maybe a major breakthrough that will vastly improve LLMs’ ability to solve general tasks is right on the horizon. These are exciting days in the world of AI!

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. DT-Mem Paper

2. Meta-World Paper

3. LoRA Paper

Let’s connect :)

Twitter | Instagram

In this post, let’s explore how it works.

Structured Data Types

There are 3 types of data considered in this approach

1. Knowledge Graph

   A set of triplets. Each triplet {x1, r, x2} represents there is a relation r between the head entity x1 and tail entity x2.

2. Data Table

   A table with c columns and r rows.

3. Database

   A set of Data Tables with some relations between the tables represented using foreign keys.

Working

Based on the 3 data types, there are 3 tasks that StructGPT can solve:

KG-based question answering (KGQA)

Setup

An intuitive approach is to start from the head entity of the query and jump along various relations that have the head as the tail of the n-1th relation. For example, if we start from with the relation {x1, r, x2}, and we also have {x2, r, x3}, we can move from x1 to x3 via x2.

The two functions used for this are:

• Extract_Neighbor_Relations(e): extracts all the neighbouring relations of the entity e.

• Extract_Triples (e, {r}): extracts all the triples with the relation in {r} and head entity e.

Implementation

1. Call Extract_Neighbor_Relations(e) to get all the one-hop relations.

2. Linearize the data to form text and feed it into the LLM to get useful relations from the set of all relations.

3. Call Extract_Tripets(e, {r}) to get all the head and tail of the useful relations.

4. Linearize into text, and feed to LLMs to get the answer which will be the tails of the most relevant triplets.

Table-based question answering (TableQA)

Setup

The goal here is for the LLMs to choose the relevant columns based on the query and then scan them row by row to get the exact sub-table required for generating the response. The methods used for this are:

• Extract_Column_Name (T): extracts all the column names of a table T.

• Extract_Columns (T, {c}): extracts the contents of columns from a table T by indices {c}.

• Extract_SubTable (T, {c}, {j}): extracts the sub-table specified by the column indices {c} and row indices {j} from a table T.

Implementation

1. Call Extract_Column_Name (T) to get all the columns.

2. Linearize them to text and feed them into the LLM to get the relevant columns.

3. Get the content of the selected columns using Extract_Columns (T, {c}).

4. Get useful rows from the column content using LLM query.

5. Use Extract_SubTable (T, {c}, {j}) to build a new table as the final input to the LLM. The response is the answer to the user query.

DB-based semantic parsing (Text-to-SQL)

Setup

The process is similar to that of the data table, however, there is an additional step of identifying which tables to use from the DB. The methods used for this are:

• Extract_Table & Column_Name (D): extracts the names of all the tables and their contained columns from the database.

• Extract_Tables_Information ({T}): extracts the table names, column names, and foreign keys from a set of tables {T}.

Implementation

1. Get the relevant table as well as relevant columns using Extract_Table & Column_Name (D).

2. Call Extract_Tables_Information ({T}) to get the data from selected tables.

3. Linearize the data and feed it into the LLM to get the answer.

Here is an illustration of the architecture given in the paper:

Conclusion

LLMs are getting better at various tasks day by day. As seen in this paper, one of the crucial tasks that seem to occur in various fine-tuning tasks is to linearize the raw data into a format that the LLM can process. With better linearization, LLMs will be able to solve more generalized tasks efficiently.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. StructGPT Paper

2. GitHub Repo

Let’s connect :)

Twitter | Instagram

Case Study 1: Predicting LTV at Airbnb

LTV stands for Customer Lifetime Value, meaning the amount of money a customer will spend once he signs up. Usually, this value is determined using historical data, but Airbnb took it a step further and built a model to predict the LTV of new listings.

Feature engineering

Airbnb used their internal tool Zipline, which allows the user to select features vetted on past projects. If a feature is unavailable, the user can create a new feature that is added to Zipline for further use. A configuration file is used to specify which features are to be picked up from the data. The LTV model had over 150 features!

Prototyping and Model Training

Rapid prototyping is essential when you have many features to choose from and many models that can be used for prediction. For feature selection, they used the pipelines available in Scikit-Learn. Here is a simple code given by the author:

transforms = []

transforms.append(
    ('select_binary', ColumnSelector(features=binary))
)

transforms.append(
    ('numeric', ExtendedPipeline([
        ('select', ColumnSelector(features=numeric)),
        ('impute', Imputer(missing_values='NaN', strategy='mean', axis=0)),
    ]))
)

for field in categorical:
    transforms.append(
        (field, ExtendedPipeline([
            ('select', ColumnSelector(features=[field])),
            ('encode', OrdinalEncoder(min_support=10))
            ])
        )
    )
    
features = FeatureUnion(transforms)

Using this pipeline as well as estimators, models can be trained quickly and iteratively.

Deployment

Airbnb uses a framework called AirFlow that automates the process of deployment. The design is as follows:

The framework needs two inputs:

• Model Config: Tells the location of training data, how to compute scores and how many resources to allocate.

• Fit and transform functions: They specify how to train the model and score the model respectively.

As we see the process follows a similar flow as described in the last post. Now let’s take a look at another case study, of Chicisimo.

Case Study 2: Using ML to Boost the fashion advice

Chicisimo’s goal was to use ML to learn the user’s fashion preferences and give them advice just like a friend would. Let’s look at how they set up their system.

Data Gathering

Chicisimo used the data provided by their users to build and update their dataset. This included objective features such as size, price, colour, type, etc as well as subjective features such as styles, occasions, etc.

They built a Social Fashion Graph to make the data actionable. This graph connected outfits, needs and people to each other. They thought of outfits as a playlist of various single items.

Since people can describe a similar outfit in various ways, they also build another tool called fashion ontology which is a multi-level descriptor of outfits or an n-ary tree.

Model Building

Chicisimo built a recommendation system on top of the fashion ontology tool. It allowed them to use the descriptors as meta-products and club similar products together. This solved the problem of the confusion caused by different product descriptions for the same product.

Conclusion

Case studies are an effective way to learn about how frameworks are applied in production. There are always a few changes required due to the specific requirements of a project. Hence I feel you need both, a framework to guide you when you start a new project and enough knowledge from case studies to understand that no framework will give you the full solution, but only a helping hand.

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. Airbnb Blog

2. Chicisimo Blog

Let’s connect :)

Twitter | Instagram

While reading the articles on ML System Design, I found that most of them start with data gathering and preparation. Although this is a crucial step, I found the first step described by Chip Huyen, particularly interesting. Hence I will start the article there.

Step 1: Setting up the project

Even before you start looking at which datasets you can use, you need to define the goals of the project. Here are a few questions you can ask:

1. Goal: What are you trying to achieve? Why is ML useful for achieving the goal?

2. User experience: What should the end UX be like? For example, DALL.E.2 is intended for everyone, whereas Generative Disco is intended for people with a background in video editing.

3. Constraints: These can be constraints regarding the resources like hardware, cost, etc. or software, like using a particular framework or platform.

4. Evaluation: How do you define the criteria to judge the performance?

Step 2: Setting up the Data Pipeline

The data pipeline is the process of collecting, preparing and doing an exploratory analysis of the data. Once this is done, you can get an idea about the important and non-important parts of the data which will be useful in feature engineering.

Here are a few things to consider:

1. Data Source: Where is the data coming from? Do we need additional data? If yes, how to generate it?

2. Data Storage: Which is the most effective and efficient form of storage?

3. Data preparation and exploration: As mentioned before, this step is necessary to get an idea about measures of central tendency, which features are useful, etc.

4. Privacy issues and Bias analysis: As ML models integrate more into our lives, this is a crucial step while using data. The data that is collected should be collected with consent and the bias if present should be removed to ensure a neutral model.

The Data Pipeline should also include a runtime component which will take real-time data and integrate it into future training iterations.

Step 3: Modelling

In addition to model selection and model training, this stage also includes feature engineering, hyper-parameter tuning, scaling, etc.

Model selection is largely based on what the goal is, and hence will depend on how well the project is set up. In addition to looking at obvious things like supervised vs unsupervised, and regression vs classification, an important thing to keep in mind is the resource constraints.

Even though it might be tempting to always choose a neural net of some sort, it is expensive to train and hard to debug. Hence it’s a good idea to give the “traditional” models like KNN, K-means, Decision trees, etc before trying out deep learning models.

Feature engineering is choosing which features your model will use to make predictions. This will depend on how much of an impact feature X has on the outcome. Methods such as PCA can be used to remove redundant features.

Hyper-parameters are meta-feature that affect the performance of the model. For example, learning rate, temperature, etc. Techniques such as grid search, random search, Bayesian optimization, etc. can be used for choosing between different hyper-parameter values.

Step 4: Deployment

For ML models, the deployment is not as simple as a deployment for a website. In addition to the software component used during deployment, the hardware consideration is just as important for initial deployment as well as scaling requirements.

Here are a few things to consider while deploying a model:

Hardware requirement: How will the model be trained? What are the hardware resources required (GPUs, TPUs, etc.) How will these resources be scaled up if required?

Software requirements: Where to deploy a model — on the user’s device (dedicated app) or a website? How do set up loops to get real-time data into the future training process?

These are the 4 main components that I learned while doing research for this article. I feel they provide a good framework to think about how we can set up a production-level ML project. If you have any suggestions or experience to improve this framework, please reply to this email or leave a comment!

That’s it for this issue. I hope you found this article interesting. Until next time!

📖Resources

1. ML System Design Book by Chip Huyen

2. Auto-ML Book by Matthias Feurer and Frank Hutter

3. Blog - What is ML system design

4. A huge playlist on youtube with tons of videos on ML System design

Let’s connect :)

Twitter | Instagram

Design

Generative AI has an algorithmic process to generate video from audio files. Below is the representation of all the steps to create a video.

These are some of the key features that I found while reading the paper:

Intervals

The process of video generation is done over user-defined intervals from a longer audio track. The users define START TIME and END TIME parameters to initialize an interval. In addition to the audio interval, the user also has to enter a high-level description of the interval which is used as the parent prompt.

An example given by the authors is after the description is entered, GPT-4 is used with the following prompt:

“In less than 5 words, describe an image for the following words {INTERVAL DESCRIPTION}.”

START & END Prompts

The video is generated by interpolating between two images that define the start and end of the interval precisely. The user gives an image as well as a text prompt. These images signify the image with which the video will start and the image where it will end.

Brainstorming area

This is an interesting add-on that provides a way for the user to generate start and end prompts based on text. Just like in text-to-image models, users can also specify a style in which the image, and hence the video should be created.

The brainstorming area auto-populates some prompts based on the response of GPT-4 on the {INTERVAL DESCRIPTION} from step 1.

Multiple intervals are then stitched to create the final video.

Examples

Conclusion

Video generation and editing have always been difficult and tedious tasks. Doing it requires deep domain knowledge in terms of various heavy applications like Adobe Premier Pro, FinalCut Pro, etc. AI can help to democratize this process by leveraging LLMs and text-to-image models. This is definitely a super early version of what future video generation and editing might look like and I am excited to see what’s next!

That’s it for this issue. I hope you found this article interesting. Until next time!

PS: Fun and sweet :)

📖Resources

1. Generative Disco Paper

Let’s connect :)

Twitter | Instagram

Design

The setup consists of 3 parts, The Eye, The Nerve and The Brain.

The Eye

The Eye is a Visual Language Model (VLM). Its goal is to capture the environment's visual information and answer related questions. The Nerve asks the questions and aims to use visual information to answer them in Natural Language.

The Nerve

The Nerve perform 2 actions

1. Asking detailed questions regarding the environment to The Eye.

2. Summarising the answers and generating the corresponding possible robot actions.

The Brain

The Brain is responsible for the following tasks:

1. Memorising the 3D environment based on the description given by The Nerve.

2. Planning future actions of the robot.

3. Control the actions of the robot based on the current scene and memory.

Execution

The pipeline has 5 stages:

1. Role Initialization

   This is a detailed prompt to set up the 3 agents.

2. Eye-Nerve Perception

   This is an iterative process where The Nerve repeatedly asks questions to The Eye to get as detailed a view of the environment as possible.

3. Brain-Nerve Collaboration

   The Brain acts as a higher-level agent and instructs the next steps to The Nerve.

4. Brain Reasoning and Control

   Using the power of LLMs, The Brain reasons about the most optimal action to take based on current inputs. (Recall Chain-of-Thoughts prompts of LLMs)

5. Brain-Human Interaction

   The Brain of the robot is also responsible to interact with the user.

Example

Initialization

Working

Conclusion

LLMs are getting more deeply integrated into various aspects of programming. As the LLMs’s ability to be effective at reasoning improves, we might get truly, fully autonomous agents, which given a task, choose the models, methods and information required to achieve it. This is an exciting time in the field of Computer Science and probably also human history!

📖Resources

1. LLM Brain Paper

That’s it for this issue. I hope you found this article interesting. Until next time!

Let’s connect :)

Twitter | Instagram

Why is HuggingGPT required?

The authors note that despite the recent advances in LLMs, the current models lack the following abilities:

1. The lack of ability to process complex data such as speech, images, videos, etc.

2. Creating, scheduling and coordinating between multiple sub-tasks.

3. Not being able to perform well at specialized tasks like image recognition in comparison to fine-tuned models such as a CNN.

The authors propose using LLMs as an interface to connect different AI models and use the LLM as a brain to plan and execute the tasks required to complete the larger goal.

Model Architecture

The proposed model has 4 stages:

• Task Planning

• Model Selection

• Task Execution

• Response Generation

Task Planning

In this stage, the goal is divided into sub-tasks using LLMs such as ChatGPT.

Each task object contains the following attributes:

• Task ID: Unique Identifier

• Task Dependencies: A list of task ids that must be completed stated before this task can be executed

• Task Type: Type of current task. Here is an image showing the tasks supported by the model

  

• Task Arguments: The data required to execute the task, like params of a function.

Model Selection

The model selection is done by using 3 things:

• The model description, given by the authors of AI models in HuggingFace.

• The task type.

• The original user query.

The models are filtered out by task type and then by the number of downloads of the models. K top models are given to HuggigGPT. It uses these models to solve a single-choice problem based on the above parameters.

Task Execution

Tasks are executed according to dependencies and the results are stored in order.

The architecture also proposed using local models in addition to HuggingFace API to improve speed and efficiency.

Response Generation

HuggingGPT uses ChatGPT to generate a response using the output of the previous 3 stages. The Output shows all the steps taken by the model to get the final result.

Example and Code

Here is an example of the model in action:

You can find the complete code to an application called Jarvis here. Let’s take look at the driver function that integrates all the steps we have seen.

def chat_huggingface(messages, openaikey = None):
    start = time.time()

    #Get the messages parsed from user query
    context = messages[:-1]
    input = messages[-1]["content"]
    logger.info("*"*80)
    logger.info(f"input: {input}")
		
		
    task_str = parse_task(context, input, openaikey).strip()
    logger.info(task_str)

    if task_str == "[]":  # using LLM response for empty task
        record_case(success=False, **{"input": input, "task": [], "reason":   "task parsing fail: empty", "op": "chitchat"})
        response = chitchat(messages, openaikey)
        return {"message": response}
    try:
        #Get tasks STAGE 1
        tasks = json.loads(task_str)
    except Exception as e:
        logger.debug(e)
        response = chitchat(messages, openaikey)
        record_case(success=False, **{"input": input, "task": task_str, "reason": "task parsing fail", "op":"chitchat"})
        return {"message": response}
    

    tasks = unfold(tasks)
    tasks = fix_dep(tasks)
    logger.debug(tasks)

    results = {}
    processes = []
    tasks = tasks[:]
    with  multiprocessing.Manager() as manager:
        d = manager.dict()
        retry = 0
        while True:
            num_process = len(processes)
            #Start Task Execution
            for task in tasks:
                dep = task["dep"]
                if len(list(set(dep).intersection(d.keys()))) == len(dep) or dep[0] == -1:
                    tasks.remove(task)
                    #STAGE 2 and STAGE 3
                    #run_task function does actual model inference and task execution i.e
                    process = multiprocessing.Process(target=run_task, args=(input, task, d, openaikey))
                    process.start()
                    processes.append(process)
            if num_process == len(processes):
                time.sleep(0.5)
                retry += 1
            if retry > 160:
                logger.debug("User has waited too long, Loop break.")
                break
            if len(tasks) == 0:
                break
        for process in processes:
            process.join()
        
        results = d.copy()
    logger.debug(results)

    #Generate Response STAGE 4
    response = response_results(input, results, openaikey).strip()

    end = time.time()
    during = end - start

    answer = {"message": response}
    record_case(success=True, **{"input": input, "task": task_str, "results": results, "response": response, "during": during, "op":"response"})
    logger.info(f"response: {response}")
    return answer

I have mentioned the different stages in the comments above. In this code, the tasks are taken from a predefined demo file. The run_task function checks the dependency array (deps) and selects a model to execute the tasks.

If you want to dive in deeper, the repo link is given above.

Conclusion

Even though programs like GitHub Co-Pilot have made good strides in AI programming, it still requires a lot of human input. To build truly autonomous coding bots we need systems that can not only write the code to the specification but also execute tasks such that they are sufficiently atomic and well coordinated.

Systems like HuggingGPT will help to make coding bots more efficient and help in rapid prototyping. Hence, at least in the short term, make programmers vastly more productive.

📖Resources

1. HuggingGPT Paper

2. Jarvis GitHub Repo

PS: An official version might be available soon! 😲

That’s it for this issue. I hope you found this article interesting. Until next time!

Let’s connect :)

Twitter | Instagram

Zero-Shot Prompting

As the name suggests, zero-shot prompting is using the model to directly reason or analyse a given input. This means you do not provide any example in the input of what the desired output should be like.

For example, if we want to generate a poem about winter, we can give the model a prompt like "Explain AI." The model can then generate a poem based on its pre-existing knowledge about winter.

The model generates an output based on its pre-existing knowledge about the task at hand. This is useful when there is no specific training data available for a particular task.

Few-shot prompting

When zero-shot prompting fails, few-shot prompting can be used to improve the output. This was proposed by Brown et. al. and it involves giving a few examples of what the desired output should look like.

Here is an example given in the paper:
Prompt: A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses the word whatpu is: We were travelling in Africa and we saw these very cute whatpus.

To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is:

Response: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles.

Here’s the prompt fed into ChatGPT

Even though the words are not part of the training data, few-shot prompting helped the LLM to reason about how the word should be used based on the context in which it appeared.

Chain-of-thought prompting

CoT prompting was introduced by Wei et. al. It involves prompting a complex series of steps to get to the desired output. Using the logic followed in the prompt, the LLM is able to solve similar problems.
For example,

Prompt:

Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can have 3 tennis balls. How many tennis balls does he have now?

Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.

The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?

Output:

The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9.

In the prompt, we provide not only the correct answer but also how we got to it. Using similar reasoning, the LLM will respond to the next question. The early versions of GPT-3 used to fail to solve such problems without explicit CoT.

CoT helps to make the LLM model reason better on various tasks such as arithmetic reasoning, common sense reasoning, instruction and planning, etc.

One note here, the authors in the paper have mentioned,

“Through experiments on arithmetic, symbolic, and commonsense reasoning, we find that chain-of-thought reasoning is an emergent property of model scale that allows sufficiently large language models to perform reasoning tasks that otherwise have flat scaling curves.“

Hence CoT might not work on smaller LLMs.

Conclusion

Advanced prompting techniques like zero-shot prompting, few-shot prompting, and chain-of-thought prompting are powerful tools in the field of machine learning. These techniques enable models to generate outputs for tasks that they have never seen before and to generate complex outputs that require multiple steps.

Learning these will help us to use LLMs efficiently and get the most out of them.

📖Resources

1. Few-shot prompting paper

2. Chain-Of-Thought prompting paper

3. Dair-ai: Prompt-Engineering-Guide

That’s it for this issue. I hope you found this article interesting. Until next time!

Let’s connect :)

Twitter | Instagram

Intuition

Large Language Models (LLMs) predict the probability of a sequence of words in a given context. In other words, given a sequence of words W0….Wn, an LLM predicts the Wn+1.

Hence the quality of the prompt and subsequently the response depends on the context created by the prompt.

Let's take a straightforward example to understand why context matters:

We want to complete the sentence, “I want to [LLM RESPONSE]”

Now with absolutely no context, you can pretty much have millions of options to choose from (Basically, anything under the sun). However, adding a simple context, “I am hungry. I want to [LLM RESPONSE]” narrows it down to just a few options.

Now even though this example was very trivial, this pattern holds true for any prompt no matter how complex the thing it is trying to achieve. The better the context that can be inferred from your prompt, the better response the model will generate.

Exploring different types of prompts

Summarization

Text summarization is probably one of the most valuable applications of LLMs. You can quickly generate a short explanation for various concepts.

To get a good summary, give prompts that have quantifiable parameters. For example, instead of the input - summarize this text, prompts like Explain to me like I’m 5 or give me an N sentence summary work much better.

I tried this with ChatGPT for one of my blogs, and here are the outputs:

You can further experiment with other prompts or other parameters. For example, the author in this post claims, 11 is a good age to get a good result from the ‘Explain to me like I’m _____ prompt’!

Text generation with the writing style

After summarization, writing quick drafts is another useful application for LLMs.

In addition to the topic of the draft, you can ask it to write a blog in a particular author’s style.

For example, write an essay in the style of X. As an example, here’s a short note on the meaning of life in the style of Elon Musk, APJ Abdul Kalam, and me!

I think it does a pretty good job of detailing the aspects which it thinks are important for that individual. In addition, notice how the prompt for me is slightly different than for famous people. I can not give a concrete reason why a different prompt is required for me, however, experimenting more with this can help us to catch such behaviours.

Testing your ideas

In addition to using the styles for drafts, you can use them to test your ideas. For example, based on a position you take, you can see if you get a useful rebuttal by using the style of an expert.

Now although this should not be taken as absolute truth, it does provide a useful way to find out opposing ideas that we might find difficult to come up with due to a position we have taken.

Coding

The most talked about and “scary” use case of ChatGPT. Even though a lot of doomsday scenarios have already been painted vividly, I feel that at least in the short term, ChatGPT is the best tool created for programmers yet.

My main use case is to quickly generate templates for whatever code I want to write and then add in the customization according to what I want to do. A word of caution here is to check the versions of the libraries that are being used in the code. I found that it will regularly use old versions even if the libraries were updated before the training cut-off.

Here’s an example of generating a simple CRUD API in NodeJS:

/*
QUERY:
Write an API in NodeJS that has separate end points for each of the CRUD operations

The APIs take a dictionary as input and perform CRUD operations on MongoDB
*/

const express = require('express');
const bodyParser = require('body-parser');
const { MongoClient, ObjectID } = require('mongodb');

const app = express();
const port = process.env.PORT || 3000;
const url = 'mongodb://localhost:27017';
const dbName = 'mydatabase';
const collectionName = 'mycollection';

app.use(bodyParser.json());

// Create
app.post('/api', async (req, res) => {
  try {
    const client = await MongoClient.connect(url);
    const db = client.db(dbName);
    const result = await db.collection(collectionName).insertOne(req.body);
    res.send(result);
    client.close();
  } catch (err) {
    res.status(500).send(err);
  }
});

// Read
app.get('/api/:id', async (req, res) => {
  try {
    const client = await MongoClient.connect(url);
    const db = client.db(dbName);
    const result = await db.collection(collectionName).findOne({ _id: new ObjectID(req.params.id) });
    res.send(result);
    client.close();
  } catch (err) {
    res.status(500).send(err);
  }
});

// Update
app.put('/api/:id', async (req, res) => {
  try {
    const client = await MongoClient.connect(url);
    const db = client.db(dbName);
    const result = await db.collection(collectionName).updateOne({ _id: new ObjectID(req.params.id) }, { $set: req.body });
    res.send(result);
    client.close();
  } catch (err) {
    res.status(500).send(err);
  }
});

// Delete
app.delete('/api/:id', async (req, res) => {
  try {
    const client = await MongoClient.connect(url);
    const db = client.db(dbName);
    const result = await db.collection(collectionName).deleteOne({ _id: new ObjectID(req.params.id) });
    res.send(result);
    client.close();
  } catch (err) {
    res.status(500).send(err);
  }
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

Another great use case is to quickly write and test regex.

/*
QUERY:
Write a regex to match all the pages on the website "<https://leetcode.com>"
*/

^https:\\/\\/leetcode\\.com\\/.*$

/*
Explanation:

^ matches the beginning of the string
https:\\/\\/leetcode\\.com\\/ matches the protocol and domain of the website
.* matches any characters that appear after the domain (i.e. any path or query parameters)
$ matches the end of the string
*/

There are many more ways to program ChatGPT for coding. Here’s a useful link that has many resources for the same.

🧾TLDR Review

Here is a compiled list of just the prompts that are explored in the article:

🔑 For Summarization use:

• Explain to me like I’m 5/11 (Try other values)

• Give me an N-point summary

🔑 For Text generation in a particular style use:

• Write [TYPE_OF_TEXT] on [TOPIC] as if written by [NAME_OF_PERSON]

• [Links to your own blogs]. Write in a similar style to me

🔑 For testing your ideas use:

• [YOUR_VIEW] Respond as if you are [PERSON_NAME] responding to the position.

🔑 For coding use:

• For Regex: Write a regex for [REQUIREMENT]

• For API templates: Write a [API_SPEC] for [API_FUNCTIONS] in [TECH_STACK_CHOICE]

Conclusion

LLMs are here to stay. They will be integrated into a myriad of different applications that we use every day. Instead of complaining and trying to put them down, it’s much more productive to try and get better at using them to improve our output as well as save time. In addition to this, the more you play with it, the more you’ll realize that we still have a long way to go to get something that will replace humans in every task!

Until then, keep experimenting with different types of prompts and exploring the various ways to make ChatGPT and other LLMs your friend!

📢Announcement

I have enabled the chat feature for Decoding Coding. Click here for the details and join the chat to let me know what you would like me to cover next!

That’s it for this issue. I hope you found this article interesting. Until next time!

Let’s connect :)

Twitter | Instagram