(Image source: University of British Columbia)

👉

This post was originally published in November 2020 by guest author Michiel Mulders. At the time, Meilisearch was on v0.15. It has been updated by Carolina Ferreira to work with Meilisearch v1.

Meilisearch JS is a client for Meilisearch written in JavaScript. Meilisearch is a powerful, fast, open-source, easy to use and deploy search engine. Both searching and indexing are highly customizable. Features such as typo-tolerance, filters, and synonyms are provided out-of-the-box.

Check the demo.

The goal of this project is to teach you how to:

Create a new index
Modify settings for your index
Add data to your index
Ignore stop words
Enable filters for faster data filtering

We’ll be using both the API directly as the JavaScript wrapper to show you how both work. Throughout this tutorial we’ll be using a cool Nobel prize winners data set to show you a couple of examples.

First, let’s take a look at the requirements.

Requirements

Here are the requirements to be able to follow this tutorial:

Node.js installation
A running self-hosted instance of Meilisearch. Don’t want to set up your own Meilisearch instance? Try out our Meilisearch cloud option!
cURL for sending requests from the terminal (Postman works as well)
Meilisearch-js wrapper and dependencies (see installation guide).

Let’s get started with the creation of your first index.

Project setup and Meilisearch-js installation

To follow along, we need to set up our JavaScript project and install Meilisearch-js. Create a new folder and run the following command in your terminal.

npm init -y

This will prepare your project setup. Next, we can add the Meilisearch-js dependency.

npm install meilisearch

Lastly, let’s create a file called index.js in your project. We’ll use this file to add our JavaScript code.

touch index.js

Good, let’s get started!

Step 1: Create your first index

First of all, let’s start with the preparation. We assume here that you have a running instance of Meilisearch and can access this via your localhost or a public IP address.

Important: To keep things simple, we aren’t using a master key. A master key allows you to protect all API endpoints for your Meilisearch instance. We highly recommend setting a master key when using Meilisearch in production or hosting it via a publicly accessible IP address - e.g. DigitalOcean droplet.

To verify if you can reach your Meilisearch instance, try querying for the available indexes. If you haven’t created any indexes, you should see an empty array as a result. Below you find the cURL command you can execute from your terminal.

curl http://127.0.0.1:7700/indexes

Now, let’s edit our index.js file to create a connection object. We’ll use this connection object to create our first index. First, we need to import the Meilisearch dependency. Furthermore, the host property accepts the IP address of your Meilisearch instance.

const { MeiliSearch } = require('meilisearch')
 
const main = async () => {
    const client = new MeiliSearch({
        host: 'http://127.0.0.1:7700'
    })
 
    const indexes = await client.getIndexes()
    console.log(indexes)
}
 
main()

Note that we’ve added some extra code that uses the client object to query for all indexes and then prints the result.

To execute the file, you can run it with the node command.

node index.js

Finally, let’s create our first index. As we are working with Nobel prizes, let’s name the index prizes. We can use the createIndex function to create a new index. To verify the successful creation of our index, let’s query again for all indexes to see the newly created index.

const { MeiliSearch } = require('meilisearch')
 
const main = async () => {
    const client = new MeiliSearch({
        host: 'http://127.0.0.1:7700'
    })
 
    const indexes = await client.getIndexes()
    console.log(indexes)
 
    const indexCreationTask = await client.createIndex('prizes')
    await client.waitForTask(indexCreationTask.taskUid)
 
    const updatedIndexes = await client.getIndexes()
    console.log(updatedIndexes)
}
 
main()

You should see the following result being printed to your terminal.

{
  results: [
    Index {
      uid: 'prizes',
      primaryKey: null,
      httpRequest: [HttpRequests],
      tasks: [TaskClient]
    }
  ],
  offset: 0,
  limit: 20,
  total: 1
}

Note: Meilisearch hasn’t set a primary key yet for the prizes index. When we add data in the next step, the primary key will be automatically picked as our data set contains an id field.

Index created? Good! Let’s explore the Nobel prizes data set.

Step 2: Adding the Nobel prizes dataset

First of all, let’s explore the dataset briefly. The original dataset we used for this example comes from nobelprize.org, however, we’ve modified the dataset slightly to fit our use case.

You can explore the modified dataset here. The structure of the data looks like this.

Each object contains an ID that will serve as the primary key. Meilisearch will automatically search for a property that ends with the string id, such as prizeId or objectId. If you want to use a dataset that doesn’t contain an id, you can still manually set a primary key.

Further, we find properties such as year, category, firstname, surname, motivation, and share.

{
id: "991",
year: "2020",
category: "chemistry",
firstname: "Emmanuelle",
surname: "Charpentier",
motivation: "for the development of a method for genome editing",
share: "2"
}

Now, let’s download the dataset as a JSON file using cURL. We are using the -o property to define an output file for the downloaded contents.

curl -L https://raw.githubusercontent.com/meilisearch/demos/main/src/nobel-prizes/setup/prizes.json -o prizes.json

Next, we need to add the dataset to the Meilisearch instance. Let’s upload the dataset to the prizes index. Note that the URL changed slightly as we are adding documents to the prizes index: indexes/prizes/documents. Make sure the filename for the --data property matches the filename of your prizes JSON file.

curl -i -X POST 'http://127.0.0.1:7700/indexes/prizes/documents' \
  --header 'content-type: application/json' \
  --data @prizes.json

To verify if the data has been uploaded successfully, let’s query for all documents. You should see all Nobel prize objects.

curl http://127.0.0.1:7700/indexes/prizes/documents

Success! Next, let’s use some code to add an extra document to our prizes index.

Step 2.1: Add documents using the Meilisearch-js client

We’ve just added documents using our terminal. It’s time to add an extra document using JS code. Let’s define an array that contains new documents we want to add to our prizes index. Note, we first need to retrieve our index so we can use this index object to add documents.

const { MeiliSearch } = require('meilisearch')
 
const main = async () => {
    const client = new MeiliSearch({
        host: 'http://127.0.0.1:7700'
    })
 
    const index = client.index('prizes')
    const documents = [
        {
            id: '12345',
            year: '2021',
            category: 'chemistry',
            firstname: 'Your',
            surname: 'Name',
            motivation: 'for the development of a new method',
            share: '1'
        }
    ]
 
    let response = await index.addDocuments(documents)
    console.log(response) 
    //  => EnqueuedTask {
    //           taskUid: 4170,
    //           indexUid: 'prizes',
    //           status: 'enqueued',
    //           type: 'documentAdditionOrUpdate',
    //           enqueuedAt: 2023-04-19T12:14:57.748Z
    //         }
}
 
main()

When you add a new document, Meilisearch returns an object containing an taskUid. Using the tasks methods you can track the document addition process until it is processed or failed.

In step 3, let’s learn how to search for documents.

Step 3: Search for Nobel prize documents

Searching for documents is pretty simple. Again, we first need to retrieve the index object. Next, we can use the index object to search for a particular query. As an example, we are looking for chemisytr to show Meilisearch’s type-tolerance.

const { MeiliSearch } = require('meilisearch')
 
const main = async () => {
    const client = new MeiliSearch({
        host: 'http://127.0.0.1:7700'
    })
 
    const index = client.index('prizes')
    const search = await index.search('chemisytr')
    console.log(search)
}
 
main()

This returns a huge list of results. Let’s learn how you can add filters to, for example, limit the number of results. Change the following line to add an object that accepts filters.

 const search = await index.search('chemisytr', { limit: 1})

This returns the following result.

{
  hits: [
     { 
       id: '991',
       year: '2020',
       category: 'chemistry',
       firstname: 'Emmanuelle',
       surname: 'Charpentier',
       motivation: '"for the development of a method for genome editing"',
       share: '2' 
     } 
  ],
  query: 'chemisytr',
  processingTimeMs: 1,
  limit: 1,
  offset: 0,
  estimatedTotalHits: 111
}

Next, we want to modify the settings for the prizes index to eliminate stop words.

Step 4: Modify index settings to eliminate stop words

Now, let’s take a look at the settings for our prizes index. You can access the settings via the exposed API like so:

curl http://localhost:7700/indexes/prizes/settings

You’ll see the following result with an empty stopWords array.

{
  displayedAttributes: [ '*' ],
  searchableAttributes: [ '*' ],
  filterableAttributes: [],
  sortableAttributes: [],
  rankingRules: [ 'words', 'typo', 'proximity', 'attribute', 'sort', 'exactness' ],
  stopWords: [],
  synonyms: {},
  distinctAttribute: null,
  typoTolerance: {
    enabled: true,
    minWordSizeForTypos: { oneTypo: 5, twoTypos: 9 },
    disableOnWords: [],
    disableOnAttributes: []
  },
  faceting: { maxValuesPerFacet: 100 },
  pagination: { maxTotalHits: 1000 }
}

We can achieve the same using JavaScript code like so.

 const index = client.index('prizes')
 const settings = await index.getSettings()
 console.log(settings)

Now, let’s add a couple of stop words we want to eliminate. Stop words are frequently occurring words that don’t have any search value.

For example, no products exist that are called a or the. To improve the search speed, we want to avoid searching for such stop words. When the user looks for the search query “a mask”, the Meilisearch engine will automatically remove the a part and look for the word mask.

In this example, we want to eliminate the following stop words:

First, let’s check how many results we receive when querying for the word the.

 const index = client.index('prizes')
 const results = await index.search('the')
 console.log(results.estimatedTotalHits)

The above query for the returns 495 hits. Now, let’s modify our index.js script to eliminate the above stop words.

 const index = client.index('prizes')
 const response = await index.updateSettings({
   stopWords: ['a', 'an', 'the']
 })
 console.log(response)

To verify the effectiveness of our settings change, let’s query again for the word the. Now, this query should return 218 results. Cool right?

Quick tip: You might have forgotten a particular stop word such as and. If you send a new updateSettings request to your API, this will overwrite the old configuration. Therefore, make sure to send the full list of stop words every time you want to make changes.

Let’s move on!

Step 5: Define filters

Filters have several use-cases, such as refining search results and creating faceted search interfaces.

Filters are most useful for numbers or enums. For example, Nobel prizes are awarded for a fixed list of categories. This makes up a great filter. The same is true for the year property.

Below you find an example snippet that adds filters for the properties year and category. You can always verify which filters have been added by consulting the settings for your index.

  const index = client.index('prizes')
  const response = await index.updateSettings({
    filterableAttributes: ['category', 'year']
  })

Step 5.1: Experimenting with filters

Now, I want to query all Nobel prize winners with the name Paul. This returns 14 results.

 const index = client.index('prizes')
 
 const search1 = await index.search('paul')
 console.log(`Search 1 hits: ${search1.estimatedTotalHits}`) // 14

Following, I want to filter down the results by the category chemistry. Note that we send an extra data property with the request that allows us to set filters. This filter property expects a filter expression containing one or more conditions.

 const index = client.index('prizes')
 
 const search2 = await index.search('paul', { filter: 'category = "Chemistry"' })
 console.log(`Search 2 hits: ${search2.estimatedTotalHits}`) ) // 5

This query returns five results.

Lastly, I want to add some extra filters to filter by both category and year. I want to return Nobel prize winners for the year 1995, 1996, or 1997. Luckily, Meilisearch allows for combining multiple conditions. You can build filter expressions by grouping basic conditions using AND and OR. Filter expressions can be written as strings, arrays, or a mix of both. Learn more about filter expressions in the documentation.

 const index = client.index('prizes')
 
 const search3 = await index.search('paul', {filter: 'category = "Chemistry" AND (year = 1995 OR year = 1996 OR year = 1997)'})
 console.log(`Search 3 hits: ${search3.estimatedTotalHits}`) // 2

Ultimately, this returns only two results that match our needs.

{ 
  hits:[ 
    { 
       id: '287',
       year: '1997',
       category: 'chemistry',
       firstname: 'Paul D.',
       surname: 'Boyer',
       motivation:
        '"for their elucidation of the enzymatic mechanism underlying the synthesis of adenosine triphosphate (ATP)"',
       share: '4' 
     },{ 
       id: '281',
       year: '1995',
       category: 'chemistry',
       firstname: 'Paul J.',
       surname: 'Crutzen',
       motivation: '"for their work in atmospheric chemistry, particularly concerning the formation and decomposition of ozone"',
       share: '3' 
     }
  ],
  offset: 0,
  limit: 20,
  nbHits: 2,
  exhaustiveNbHits: false,
  processingTimeMs: 0,
  query: 'paul' 
}

Nice! Lastly, let’s play with ranking rules for our Meilisearch engine.

Step 6: Define your own ranking rule

In step 3, we’ve shown you how Meilisearch handles typos by querying for chemisytr instead of chemistry.

However, you might have noticed that the settings for your index list many different ranking rules. Ranking rules are what defines relevancy in Meilisearch. They affect how a result is considered more relevant than another. Ranking rules are sorted top-down by order of importance.

It’s possible to define your own ranking rule. That’s pretty exciting right? Let’s add a custom ranking rule for the year property.

You can define an ascending or descending sorting rule.

    const index = client.index('prizes')
    await index.updateSettings({
        rankingRules:
            [
                "year:desc",
                "words",
                "typo",
                "proximity",
                "attribute",
                "sort",
                "exactness"
            ]
    })

Next, let’s search for Paul again. Now, notice that the results are sorted by the year property as expected.

[ 
  { 
    id: '995',
    year: '2020',
    category: 'economics',
    firstname: 'Paul',
    surname: 'Milgrom',
    motivation: '"for improvements to auction theory and inventions of new auction formats"',
    share: '2'
   },
   { 
     id: '834',
     year: '2008',
     category: 'economics',
     firstname: 'Paul',
     surname: 'Krugman',
     motivation: '"for his analysis of trade patterns and location of economic activity"',
     share: '1'
   },
   { 
     id: '764',
     year: '2003',
     category: 'medicine',
     firstname: 'Paul C.',
     surname: 'Lauterbur',
     motivation: '"for their discoveries concerning magnetic resonance imaging"',
     share: '2'
   },
   …
]

That’s it!

Conclusion

That’s it for this Meilisearch and JS tutorial. This tutorial has taught you how to use the Meilisearch API, create indexes, modify index settings, and define filters for more accurate and faster searching.

Try out our live demo.

For more information, make sure to take a look at the documentation and the JS API wrapper on GitHub.

Liked using Meilisearch, make sure to show us some love by giving Meilisearch a star on GitHub!

How to search Nobel Prize winners faster with Meilisearch and JavaScript

Michiel Mulders

Carolina Ferreira

Michiel Mulders, Carolina Ferreira

Requirements

Project setup and Meilisearch-js installation

Step 1: Create your first index

Step 2: Adding the Nobel prizes dataset

Step 2.1: Add documents using the Meilisearch-js client

Step 3: Search for Nobel prize documents

Step 4: Modify index settings to eliminate stop words

Step 5: Define filters

Step 5.1: Experimenting with filters

Step 6: Define your own ranking rule

Conclusion

Meilisearch 1.7

A complete guide to full-text search with Laravel

Introducing Teams — Meilisearch Cloud

What's new in v0.16.0

What's new in v0.15.0