OpenSearch for Confluence: Improving Team Knowledge Discovery

Step-by-Step Guide: Migrating Confluence Search to OpenSearch

Overview

This guide walks through migrating Confluence Data Center from the default Lucene search to OpenSearch with minimal downtime. Assumes Confluence 9.0+ and an OpenSearch cluster (OpenSearch 2.11–2.19 supported). Default approach: clustered deployment with a dark node to avoid search downtime. If you run a single-node instance, use the “simple” method (downtime required).

Pre-migration checklist

• Confluence version ≥ 9.0 (verify compatibility).
• Backup Confluence (database + home directory) and application nodes.
• Provision OpenSearch cluster (self-managed or managed service). Secure it (TLS, auth).
• Confirm OpenSearch version compatibility (2.11–2.19).
• Ensure hardware/network sizing for indexing load.
• Identify times for maintenance windows and stakeholders.
• Verify any plugins/custom code that use Lucene APIs — update to Confluence Search v2 API if needed.
• Read Confluence release notes for OpenSearch-related behavior changes (result window limits, field-mapping consistency).

Option A — Clustered migration without downtime (recommended)

Applicable when you have a Confluence Data Center cluster with at least two nodes.

1. Provision a dark Confluence node

   • Create a new Confluence node and do not add it to the load balancer / user traffic group.
   • Label it “dark node”.

2. Configure the dark node to use OpenSearch

   • Provision or confirm your OpenSearch cluster endpoint and credentials.
   • Set the search platform to OpenSearch:
     • In confluence.cfg.xml or as a system property set search.platform = opensearch.
     • Provide OpenSearch hosts/auth info via recognized system properties (or confluence.cfg.xml). Example properties (use secure storage for credentials):
       • opensearch.hosts
       • opensearch.username / opensearch.password or other auth method
   • Start the dark node and confirm it connects to OpenSearch (System Information page -> search engine shows OpenSearch).

3. Rebuild the index on the dark node

   • In the dark node, go to Administration → Content Indexing → Rebuild index (or use the Re-index admin UI).
   • Monitor index build progress and logs. Reindexing duration depends on content size.

4. Verify search on the dark node

   • On the dark node, perform sample searches to confirm content returns as expected.
   • Check logs for mapping conflicts or errors (Confluence logs and OpenSearch logs).

5. Promote OpenSearch across the cluster

   • Once the OpenSearch index is fully rebuilt and verified:
     • Reconfigure remaining nodes to use OpenSearch (set search.platform = opensearch and OpenSearch connection properties).
     • Start/restart nodes and verify they report OpenSearch in System Information.
   • Gradually add these nodes into the load balancer; drain old Lucene nodes and remove them.

6. Decommission dark node (optional)

   • After all nodes run OpenSearch and users are satisfied, remove the dark node or repurpose it.

7. Post-migration checks

   • Confirm search functionality for common queries and advanced queries (CQL).
   • Validate permissions, content restrictions, attachments, and custom indices.
   • Monitor OpenSearch cluster health and Confluence logs for errors.
   • Adjust OpenSearch index settings if necessary (index.max_result_window, analyzers, refresh interval).

Option B — Simple migration with downtime (single-node or accept downtime)

1. Backup Confluence.
2. Configure Confluence to use OpenSearch (confluence.cfg.xml or system properties).
3. Reconfigure OpenSearch connection and restart Confluence so it connects to OpenSearch.
4. Rebuild the index via Administration → Content Indexing → Rebuild index.
5. Wait until indexing completes, verify searches, then resume normal operations.

Note: During rebuild, search may be unavailable or incomplete.

Handling specific issues & compatibility notes

• Field mapping conflicts: OpenSearch requires consistent field mappings across documents. Update any custom indexers/plugins to use consistent mappings or the Confluence v2 Search API.
• Result window limits: OpenSearch default max_result_window = 10000. If apps rely on deep pagination, refactor to use search_after or increase index.max_resultwindow (be cautious of memory impact).
• Custom plugin indices: Implement a ReIndexRequestEvent listener for custom indices so they rebuild during admin-triggered reindex.
• Attachments and binary extraction: Ensure any extractors/external indexing tooling follow Confluence guidance for OpenSearch.
• Security: Use TLS and proper auth (basic, proxy, or managed identity). If using AWS OpenSearch Service, follow its recommended IAM role setup and networking rules.