Boosting MySQL with Vector Search Fosdem 2025.pdf
0 likes92 views
The document introduces the MyVector plugin for MySQL, which enhances the database's capabilities by enabling vector search through the utilization of vector embeddings. This allows for improved relevance in search results, supporting various AI-based applications and advanced data analysis. The MyVector plugin features a straightforward architecture for integration and administration, facilitating similarity searches and the handling of high-dimensional data.
![MySQL Vector Plugin Examples
myvector_construct(vector_string VARCHAR):
Purpose: Converts a human-readable vector string into a serialized binary
format suitable for storage in a VARBINARY / VECTOR column.
Usage Example:
INSERT INTO vectors_table (vector_column)
VALUES (myvector_construct('[0.1, 0.2, 0.3, ...]'));
@2025 Altinity, Inc.](https://image.slidesharecdn.com/boostingmysqlwithvectorsearchfosdem2025-250205092818-8264c1d3/85/Boosting-MySQL-with-Vector-Search-Fosdem-2025-pdf-21-320.jpg)
![MySQL Vector Plugin Examples
Similarity Search (ANN)
SELECT <column-list> FROM <table> WHERE
MYVECTOR_IS_ANN('vector_column','key column',
<search vector>, options)
// search vector(s) should first be inserted to <query_table>
SELECT <column-list> FROM
MYVECTOR_SEARCH[<base_table>, <key column>,
<vector_column>, <query_table>]
@2025 Altinity, Inc.](https://image.slidesharecdn.com/boostingmysqlwithvectorsearchfosdem2025-250205092818-8264c1d3/85/Boosting-MySQL-with-Vector-Search-Fosdem-2025-pdf-24-320.jpg)
![MySQL Vector Plugin Examples
myvector_ann_set(veccol VARCHAR, options VARCHAR, searchvec VECTOR/VARBINARY):
Purpose: Returns a comma-separated list of IDs corresponding to the nearest neighbors of a given search
vector.
Usage Example:
SELECT myvector_ann_set('vector_column', 'options', myvector_construct('[0.1, 0.2, 0.3,
...]')) AS neighbors FROM vectors_table;
@2025 Altinity, Inc.](https://image.slidesharecdn.com/boostingmysqlwithvectorsearchfosdem2025-250205092818-8264c1d3/85/Boosting-MySQL-with-Vector-Search-Fosdem-2025-pdf-25-320.jpg)
Boosting MySQL with Vector Search Fosdem 2025.pdf
- 1. Boosting MySQL with Vector Search Fosdem 2025 Introducing the MyVector Plugin Alkin Tezuysal / Shankar Iyer
- 2. Let’s get connected! Alkin Tezuysal - Director of Services @AltinityDB ● Linkedin : https://www.linkedin.com/in/askdba/ Open Source Database Evangelist ● Previously ChistaDATA, PlanetScale, Percona and Pythian as Senior Technical Manager, SRE, DBA ● Earlier in life Enterprise DBA , Informix, Oracle, DB2 , SQL Server ● Recent Recognitions: ○ Most Influential in Database Community 2022 - The Redgate 100 ○ MySQL Cookbook, 4th Edition 2022 - O'Reilly Media, Inc. ○ MySQL Rockstar 2023 - Oracle (MySQL Community) ○ Database Design and Modeling with PostgreSQL and MySQL 2024 - <Packt> @ask_dba @2025 Altinity, Inc. Born to Sail, Forced to Work!
- 3. Sailing Trivia 1 nautical mile (nm) equals to …
- 4. Let’s get connected! Shankar Iyer ● Linkedin : https://www.linkedin.com/in/shankar-iyer-a4a1 563/ ● Earlier in life : Engineer in Oracle RAC Database Development Team (link) ● Recently worked in AWS Aurora MySQL, Google Cloud MySQL, Clickhouse ● Interests : MySQL for GenAI, Research & Experimentations in MySQL performance/replication/availability, SQL optimizations for BI -
- 5. Vectors and Dimensions ● Mathematical objects representing magnitude and direction (velocity, force) ● Vectors can be represented as arrays or ordered lists of numbers ○ 2D (x, y), 3D (x, y, z), n-dimensional (AI) ● Basically data points ● When dimensions increase data points increase and they turn into embeddings @2025 Altinity, Inc.
- 6. Understanding Vector Embeddings ● Vector Embeddings: Mathematical representations of data in a high-dimensional space. ● Capture semantic meaning and relationships between words, phrases, or other data. ● Enable more accurate and relevant search results compared to traditional keyword search. ● Power AI applications like semantic search, recommendation systems, and chatbots. ● Data analysis and application functionality @2025 Altinity, Inc.
- 7. Vector Embeddings Transforming data into vectors in a way that captures the semantic meaning or relationships between the data points. (The image on the left shows a OpenAI vector embedding for a Wikipedia article using text-embedding-ada-002 Vector dimension is 1536) @2025 Altinity, Inc. OpenAI has another model text-embedding-3-large model provides 3,072-dimensional vector embeddings. P.S: Deep Seek can generate embedding dimensions upto 8K!
- 8. Benefits of Vector Search ● Efficient and relevant search results. ● Improved relevance for text and image search. ● Semantic understanding. ● Supports AI-based applications. ● Leverage and unlock value from live and historical business data in the MySQL database : e.g conversation logs, clinical records, support tickets, persona descriptions, medical images, product catalog, legal filings etc. @2025 Altinity, Inc.
- 9. Vector Support Helps ● Traditional search struggles with high-dimensional data and semantic understanding. ● Inefficient for AI and ML applications that rely on vector embeddings. ● Vector support enhances performance and relevance for similarity searches. ● Enables new AI-driven applications directly within MySQL. ● Unlocks advanced data analysis techniques using vector embeddings. @2025 Altinity, Inc.
- 10. Why Vector Search in MySQL? ● MySQL World’s number one open-source relational database ● Vector search allows for more efficient and relevant search results. ● Improved relevance for text and image search with semantic understanding. ● MyVector plugin extends MySQL with vector search capabilities. ● Supports a wide range of AI-based applications. P.S: MySQL does not come with extensibility interfaces to add new data types, and no support to add new index types and to add new access methods @2025 Altinity, Inc.
- 11. Vector Search Use Cases Use Cases: Natural Language Processing (NLP): Words or sentences are converted into vectors where similar words or phrases are closer in vector space. Recommender Systems: Items or users can be embedded to find similarities or predict preferences. Image Recognition: Images can be encoded into vectors for tasks like similarity search or classification. @2025 Altinity, Inc.
- 12. Vector Databases Vector Database Providers ● Pinecone ● Qdrant ● Milvus ● Weavite ● Chroma Vector Store Providers ● ClickHouse ● Elasticsearch ● Singlestore ● Typesense ● Others ○ MySQL ○ MariaDB ○ PostgreSQL. @2025 Altinity, Inc.
- 13. Vector Databases vs Traditional Databases @2025 Altinity, Inc.
- 14. MySQL Vector Plugin Architecture The MyVector Plugin, discussed earlier, leverages this architecture. Here's how it fits: ● UDFs in the plugin: Adds vector-related SQL functions. ● Data storage : Use varbinary in MySQL 8.x and VECTOR in MySQL 9.x. ● Indexing: Integrates vector similarity search via HNSW indexing. ● Similarity Search - Intuitive syntax for search using SQL ● Administration - Simple MySQL stored procedures @2025 Altinity, Inc.
- 15. MySQL Vector Plugin Build /// get the MyVector sources $ cd mysql-server/src/plugin $ git clone https://github.com/p3io/myvector-dev/ ./myvector /// Generate makefile for the new plugin $ cd mysql-server/bld $ cmake .. <other options used for this build > /// Build the plugin $ cd mysql-server/bld/plugin/myvector $ make @2025 Altinity, Inc.
- 16. MySQL Vector Plugin Install /// Copy the MyVector plugin shared library to the MySQL installation plugins $ cp mysql-server/bld/plugin_output_directory/myvector.so /usr/local/mysql/lib/plugin/ /// Register the MyVector plugin and create MyVector stored procedures. $ cd mysql-server/plugin/myvector /// Connect to 'mysql' database as 'root' $ mysql -u root -p mysql mysql> source myvectorplugin.sql @2025 Altinity, Inc.
- 17. MySQL Vector Plugin Architecture ● MySQL V8.X - InnoDB + Plugin + Varbinary Data Type ● MySQL V9.X - InnoDB + Plugin + Vector Data Type mysql> select udf_name from performance_schema.user_defined_functions where udf_name like "myvector%" ; +-----------------------------+ | udf_name | +-----------------------------+ | | | myvector_is_valid | | myvector_distance | | myvector_display | | myvector_ann_set | | myvector_row_distance | | myvector_construct | +-----------------------------+ @2025 Altinity, Inc.
- 18. MyVector Features High speed, parallel build of HNSW index Specify recall v/s latency parameters : M, ef, ef_search HNSW index is incrementally persisted after initial build Online update of index via binlog read & parse of DMLs HNSW Index is crash-safe and is recovered on MySQL instance crash Write amplification to redo log/undo log avoided @2025 Altinity, Inc.
- 19. MySQL Vector Plugin Examples MySQL 8.X Ref: https://nlp.stanford.edu/projects/glove/ @2025 Altinity, Inc. mysql > create table words50d( wordid int primary key, word varchar(200), wordvec MYVECTOR(type=HNSW, dim=50,size=400000, M=64,ef=100) ); – Load the Data (insert …) mysql> call myvector_index_build ('test.words50d.wordvec','wordid');
- 20. MySQL Vector Plugin Examples MySQL 9.1 mysql > create table words50d( wordid int primary key, word varchar(200), wordvec MYVECTOR(type=HNSW, dim=50,size=400000, M=64,ef=100) ); – Load the Data (insert …) mysql> call myvector_index_build ('test.words50d.wordvec','wordid'); @2025 Altinity, Inc.
- 21. MySQL Vector Plugin Examples myvector_construct(vector_string VARCHAR): Purpose: Converts a human-readable vector string into a serialized binary format suitable for storage in a VARBINARY / VECTOR column. Usage Example: INSERT INTO vectors_table (vector_column) VALUES (myvector_construct('[0.1, 0.2, 0.3, ...]')); @2025 Altinity, Inc.
- 22. MySQL Vector Plugin Examples myvector_display(vector_col_expr VARBINARY/VECTOR): Purpose: Transforms a binary-stored vector back into a human-readable string representation. Usage Example: SELECT myvector_display(vector_column) AS readable_vector FROM vectors_table; @2025 Altinity, Inc.
- 23. MySQL Vector Plugin Examples myvector_distance(vec1 VARBINARY/VECTOR, vec2 VARBINARY/VECTOR, disttype VARCHAR): Purpose: Calculates the distance between two vectors using the specified distance metric ('L2', 'EUCLIDEAN', or 'IP' for inner product). Usage Example: mysql> select myvector_distance((select wordvec from words50d where word = 'school'), (select wordvec from words50d where word='institute'))G *************************** 1. row *************************** myvector_distance((select wordvec from words50d where word = 'school'), (select wordvec from words50d where word='institute')): 25.51254653930664 1 row in set (0.36 sec) @2025 Altinity, Inc.
- 24. MySQL Vector Plugin Examples Similarity Search (ANN) SELECT <column-list> FROM <table> WHERE MYVECTOR_IS_ANN('vector_column','key column', <search vector>, options) // search vector(s) should first be inserted to <query_table> SELECT <column-list> FROM MYVECTOR_SEARCH[<base_table>, <key column>, <vector_column>, <query_table>] @2025 Altinity, Inc.
- 25. MySQL Vector Plugin Examples myvector_ann_set(veccol VARCHAR, options VARCHAR, searchvec VECTOR/VARBINARY): Purpose: Returns a comma-separated list of IDs corresponding to the nearest neighbors of a given search vector. Usage Example: SELECT myvector_ann_set('vector_column', 'options', myvector_construct('[0.1, 0.2, 0.3, ...]')) AS neighbors FROM vectors_table; @2025 Altinity, Inc.
- 26. MySQL Vector Plugin Examples Vector Index Administration - Stored Procedures // build the index CALL myvector_index_build('vector column', 'key column') // check current status of the index - number of rows, parameters etc CALL myvector_index_status('vector column') // open/load the index into memory after a restart (if not marked online) CALL myvector_index_load('vector column') // drop the the index CALL myvector_index_drop('vector column') @2025 Altinity, Inc.