Installing and Configuring Elasticsearch 8 and OpenSearch on Ubuntu 24.04 LTS for Magento 2.4.7 Search Catalogs

Prerequisites and Initial Setup for Ubuntu 24.04 LTS

This guide assumes you have a fresh Ubuntu 24.04 LTS (Noble Numbat) server instance. Ensure you have root or sudo privileges. Before proceeding, update your package lists and upgrade existing packages to their latest versions. This is crucial for security and compatibility.

sudo apt update && sudo apt upgrade -y

Next, we need to install Java, as both Elasticsearch and OpenSearch require it to run. OpenJDK 17 is the recommended version for Elasticsearch 8.x and is compatible with OpenSearch.

sudo apt install -y openjdk-17-jre-headless

Verify the Java installation by checking its version.

java -version

Installing Elasticsearch 8.x on Ubuntu 24.04 LTS

Elasticsearch 8.x introduces significant security enhancements, including mandatory TLS encryption and API key authentication. We’ll configure it for a single-node setup suitable for development or smaller production environments. For production, consider a multi-node cluster.

First, add the Elasticsearch APT repository. You’ll need to import the signing key and add the repository source.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Update your package list again to include the new repository, then install Elasticsearch.

sudo apt update
sudo apt install -y elasticsearch

Elasticsearch is installed as a systemd service. Start it and enable it to run on boot.

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service

Check the status of the Elasticsearch service.

sudo systemctl status elasticsearch.service

Once the service is active, you can test the connection using curl. Elasticsearch 8.x defaults to HTTPS and requires authentication. The initial setup generates a certificate and a password for the ‘elastic’ user.

curl -k -u elastic https://localhost:9200

You will be prompted for the ‘elastic’ user’s password. This password can be found in the Elasticsearch logs or can be reset. To retrieve the initial password:

sudo grep -a 'elastic' /var/log/elasticsearch/log-20*.log | grep 'password'

Alternatively, you can reset the password using the Elasticsearch API. First, ensure you have the correct certificate path. The default keystore location is typically /etc/elasticsearch/certs/http_ca.crt.

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i

After resetting, use the new password with the curl command.

Configuring Elasticsearch 8.x for Magento 2.4.7

Magento 2.4.7 requires specific configurations for Elasticsearch. The primary configuration file is /etc/elasticsearch/elasticsearch.yml. We need to ensure network settings, cluster name, and discovery are correctly set.

Edit the Elasticsearch configuration file:

sudo nano /etc/elasticsearch/elasticsearch.yml

For a single-node setup, ensure the following settings are present or uncommented and adjusted:

cluster.name: "magento-cluster"
node.name: "node-1"
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/http.key
xpack.security.http.ssl.certificate: certs/http.crt
xpack.security.http.ssl.certificate_authorities: [ "certs/http_ca.crt" ]
xpack.security.transport.ssl.verification_mode: certificate
cluster.routing.allocation.disk.threshold_enabled: false
indices.query.bool.max_clause_count: 2048

Important Notes on Security:

• network.host: 0.0.0.0: This binds Elasticsearch to all network interfaces. For production, restrict this to specific IP addresses or use a firewall.
• xpack.security.* settings: These are crucial for enabling TLS and authentication. Magento 2.4.7 requires these to be enabled.
• cluster.routing.allocation.disk.threshold_enabled: false: This setting prevents Elasticsearch from stopping indexing if disk space is low. While convenient, monitor disk usage closely in production.
• indices.query.bool.max_clause_count: 2048: Magento often requires a higher clause count than the default for complex searches.

After modifying elasticsearch.yml, restart Elasticsearch to apply the changes.

sudo systemctl restart elasticsearch.service

Verify the service status and test connectivity again. You will need to provide the ‘elastic’ user’s password.

curl -k -u elastic https://localhost:9200

Installing OpenSearch 2.x on Ubuntu 24.04 LTS

OpenSearch is a fork of Elasticsearch and is also a viable option for Magento 2.4.7 search. The installation process is similar, but uses a different APT repository and package name.

Add the OpenSearch APT repository. Note that OpenSearch uses a different signing key and repository URL.

wget -qO - https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo apt-key add -
echo "deb https://artifacts.opensearch.org/releases/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list

Update your package list and install OpenSearch.

sudo apt update
sudo apt install -y opensearch

Start and enable the OpenSearch service.

sudo systemctl daemon-reload
sudo systemctl enable opensearch.service
sudo systemctl start opensearch.service

Check the status of the OpenSearch service.

sudo systemctl status opensearch.service

OpenSearch 2.x also defaults to HTTPS and requires authentication. The initial setup generates a certificate and an admin user password. Test connectivity:

curl -k -u admin https://localhost:9200

You’ll be prompted for the ‘admin’ user’s password. Retrieve it from the OpenSearch logs:

sudo grep -a 'admin' /var/log/opensearch/server.log | grep 'password'

Or reset it using the OpenSearch security admin tool:

sudo /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh -cd /usr/share/opensearch/plugins/opensearch-security/securityconfig/ -icl -nhnv -cacert /usr/share/opensearch/config/root-ca.pem -cert /usr/share/opensearch/config/admin-cert.pem -key /usr/share/opensearch/config/admin-key.pem -p 9300 -h localhost -cn opensearch -u admin -r

Follow the prompts to set a new password for the ‘admin’ user. Then, use this new password with the curl command.

Configuring OpenSearch 2.x for Magento 2.4.7

The configuration file for OpenSearch is /etc/opensearch/opensearch.yml. Similar to Elasticsearch, we need to adjust network, cluster, and discovery settings.

Edit the OpenSearch configuration file:

sudo nano /etc/opensearch/opensearch.yml

For a single-node setup, ensure these settings are present or adjusted:

cluster.name: "magento-cluster"
node.name: "node-1"
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
plugins.security.disabled: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enabled: true
plugins.security.ssl.http.pemcertchainfile: certs/http_ca.crt
plugins.security.ssl.http.pemkeyfile: certs/http.key
plugins.security.ssl.transport.pemcertchainfile: certs/transport_ca.crt
plugins.security.ssl.transport.pemkeyfile: certs/transport.key
plugins.security.authcz.admin_dn: "CN=admin,OU=opensearch,O=org,L=test,C=de"
cluster.routing.allocation.disk.threshold_enabled: false
indices.query.bool.max_clause_count: 2048

Important Notes on Security:

• network.host: 0.0.0.0: As with Elasticsearch, restrict this in production.
• plugins.security.* settings: These enable the security features for OpenSearch.
• plugins.security.authcz.admin_dn: This DN must match the certificate used by the admin user.
• cluster.routing.allocation.disk.threshold_enabled: false and indices.query.bool.max_clause_count: 2048: These are important for Magento compatibility and performance.

After modifying opensearch.yml, restart OpenSearch.

sudo systemctl restart opensearch.service

Verify the service status and test connectivity. You will need to provide the ‘admin’ user’s password.

curl -k -u admin https://localhost:9200

Magento 2.4.7 Configuration for Elasticsearch/OpenSearch

Once your Elasticsearch or OpenSearch instance is running and configured, you need to inform Magento about its location and credentials. This is done via Magento’s configuration settings, typically managed through the command line or the Admin Panel.

Using the Command Line:

Navigate to your Magento root directory and run the following commands. Replace placeholders with your actual Elasticsearch/OpenSearch details.

# For Elasticsearch 8.x
bin/magento setup:config:set --elasticsearch-host=localhost --elasticsearch-port=9200 --elasticsearch-encryption=1 --elasticsearch-username=elastic --elasticsearch-password='YOUR_ELASTIC_PASSWORD' --elasticsearch-index-prefix=magento

# For OpenSearch 2.x
bin/magento setup:config:set --elasticsearch-host=localhost --elasticsearch-port=9200 --elasticsearch-encryption=1 --elasticsearch-username=admin --elasticsearch-password='YOUR_ADMIN_PASSWORD' --elasticsearch-index-prefix=magento

After updating the configuration, you must reindex your catalog search. This process will populate Elasticsearch/OpenSearch with your product data.

bin/magento indexer:reindex catalogsearch_fulltext

Finally, clear the Magento cache.

bin/magento cache:clean
bin/magento cache:flush

Troubleshooting Common Issues

Connection Refused:

• Ensure the Elasticsearch/OpenSearch service is running: sudo systemctl status elasticsearch.service or sudo systemctl status opensearch.service.
• Verify network.host in the configuration file is set to 0.0.0.0 or the correct IP address.
• Check firewall rules (e.g., ufw) to ensure port 9200 is open.

Authentication Errors (401 Unauthorized):

• Double-check the username (‘elastic’ or ‘admin’) and password.
• Ensure the password was correctly retrieved or reset.
• If using custom certificates, verify they are correctly configured and trusted.

Index Creation Failures:

• Check Elasticsearch/OpenSearch logs for specific error messages.
• Ensure indices.query.bool.max_clause_count is set to a sufficient value (e.g., 2048).
• Verify sufficient disk space on the server.

Magento Admin Panel Issues:

• After making configuration changes in Magento, always reindex and clear the cache.
• Ensure the Magento application user has the necessary permissions to communicate with Elasticsearch/OpenSearch.