data

package
v0.0.22 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2025 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	TableDataExtension   = ".dat"
	TableMetaExtension   = ".meta"
	DirectoryPermissions = 0755
	FilePermissions      = 0644
)
View Source 
const (
	Table1Prefix = "t1."
	Table2Prefix = "t2."
	NumPrefix    = "num:"
	StrPrefix    = "str:"
)

Constants for table prefixes and value types

View Source 
const (
	WorstSelectivity = 1.0
	DefaultLimit     = 0 // No limit
	DefaultOffset    = 0 // No offset
)

Constants for query execution

Variables

This section is empty.

Functions

func ValidFilename added in v0.0.12 

func ValidFilename(name string) bool

Types

type CacheEntry added in v0.0.22 

type CacheEntry struct {
	Record     *dbdata.Record
	AccessTime int64
}

Table is a struct that represents a table in the database. It includes a mutex for read-write locking to ensure thread safety during operations. FilePath is the path to the file where the table data is stored. PrimaryKey is the field name that is used as the primary key for the table. utils is a utility object used for various helper functions. Indexes is a map where the keys are field names and the values are slices of records that have that field. Records is a map where the keys are primary key values and the values are the corresponding records. CacheEntry represents a cached record with access time

type Database 

type Database struct {
	sync.RWMutex                   // Mutex to ensure the database is thread safe
	Name         string            // Name of the database
	Tables       map[string]*Table // Map of Tables in the database
}

func NewDatabase 

func NewDatabase(name string) *Database

func (*Database) CreateTable 

func (db *Database) CreateTable(tableName, primaryKey string) error

func (*Database) ListTables 

func (db *Database) ListTables() ([]string, error)

ListTables returns a list of table names in the database

func (*Database) LoadTables 

func (db *Database) LoadTables(dbDir string) error

LoadTables loads all tables from the database directory

type DatabaseReader 

type DatabaseReader interface {
	ServeHTTP(w http.ResponseWriter, r *http.Request)
	ListTables(w http.ResponseWriter) []string
}

type ExecutionPlan 

type ExecutionPlan struct {
	IndexToUse string         // Index to be used for the query
	Filters    map[string]any // Filters to be applied
	SortBy     string         // Field to sort results by
	Limit      int            // Maximum number of results
	Offset     int            // Number of results to skip
}

ExecutionPlan represents the optimized execution plan for a database query

type JoinResult added in v0.0.22 

type JoinResult map[string]any

JoinResult represents a joined record with fields from both tables

func JoinTables 

func JoinTables(t1, t2 *Table, key1, key2 string, joinType JoinType) ([]JoinResult, error)

JoinTables performs a join operation between two tables based on specified keys and join type. Supports inner, left, right, and full outer joins.

type JoinType 

type JoinType int
const (
	InnerJoin JoinType = iota
	LeftJoin
	RightJoin
	FullOuterJoin
)

type Metrics added in v0.0.12 

type Metrics struct {
	sync.RWMutex
	InsertCount int       // The number of insert operations performed.
	UpdateCount int       // The number of update operations performed.
	DeleteCount int       // The number of delete operations performed.
	QueryCount  int       // The number of query operations performed.
	CacheHits   int       // The number of successful cache retrievals.
	CacheMisses int       // The number of unsuccessful cache retrievals.
	LastInsert  time.Time // The timestamp of the last insert operation.
	LastUpdate  time.Time // The timestamp of the last update operation.
	LastDelete  time.Time // The timestamp of the last delete operation.
	LastQuery   time.Time // The timestamp of the last query operation.
}

Metrics is a structure that holds various counts and timestamps related to database operations and cache usage.

func NewMetrics added in v0.0.12 

func NewMetrics() *Metrics

NewMetrics creates and returns a new Metrics structure.

func (*Metrics) IncrementCacheHits added in v0.0.12 

func (m *Metrics) IncrementCacheHits()

IncrementCacheHits increases the count of successful cache retrievals.

func (*Metrics) IncrementCacheMisses added in v0.0.12 

func (m *Metrics) IncrementCacheMisses()

IncrementCacheMisses increases the count of unsuccessful cache retrievals.

func (*Metrics) IncrementDeleteCount added in v0.0.12 

func (m *Metrics) IncrementDeleteCount()

IncrementDeleteCount increases the count of delete operations and updates the timestamp of the last delete operation.

func (*Metrics) IncrementInsertCount added in v0.0.12 

func (m *Metrics) IncrementInsertCount()

IncrementInsertCount increases the count of insert operations and updates the timestamp of the last insert operation.

func (*Metrics) IncrementQueryCount added in v0.0.12 

func (m *Metrics) IncrementQueryCount()

IncrementQueryCount increases the count of query operations and updates the timestamp of the last query operation.

func (*Metrics) IncrementUpdateCount added in v0.0.12 

func (m *Metrics) IncrementUpdateCount()

IncrementUpdateCount increases the count of update operations and updates the timestamp of the last update operation.

func (*Metrics) String added in v0.0.12 

func (m *Metrics) String() string

String returns a string representation of the Metrics structure in JSON format.

type Query 

type Query struct {
	Filters map[string]any // Filters to select specific records
	SortBy  string         // Field to sort the records by
	Limit   int            // Maximum number of records to return
	Offset  int            // Number of records to skip (for pagination)
}

Query represents a database query with filters, sorting, and pagination options

type QueryResult added in v0.0.22 

type QueryResult struct {
	Records []Record
	Count   int
}

QueryResult holds the results and metadata of a query execution

type Record 

type Record map[string]interface{}

type Server 

type Server struct {
	sync.RWMutex                      // Mutex to ensure the server is thread safe
	Databases    map[string]*Database // Map of Databases in the server
}

func NewServer 

func NewServer() *Server

NewServer creates a new Server instance. It initializes the Databases field as an empty map where the key is a string representing the database name and the value is a pointer to a Database instance. It returns a pointer to the newly created Server instance.

func (*Server) BackupDatabases added in v0.0.11 

func (s *Server) BackupDatabases() (string, error)

BackupDatabases is a method of the Server struct that creates a backup of all databases in the server. It returns the path to the backup file and an error if there is one.

The method works as follows:

  1. It acquires a read lock on the Server struct and defers the unlocking of the lock.
  2. It creates a backup directory in the default backup directory. The default backup directory is determined by the getDefaultBackUpDir function. If there is an error creating the backup directory, the error is returned.
  3. It creates a backup file in the backup directory. The backup file is a zip file named "backup.zip". If there is an error creating the backup file, the error is returned.
  4. It creates a new zip writer for the backup file and defers the closing of the zip writer.
  5. It iterates over each database in the Databases field of the Server struct. For each database, it walks the database directory and adds each file to the zip file. The database directory is determined by the getDefaultServerDir function and the database name. If there is an error walking the database directory or adding a file to the zip file, the error is returned.
  6. If all databases are successfully backed up, the method returns the path to the backup file and nil.

func (*Server) CreateDatabase 

func (s *Server) CreateDatabase(name string) error

CreateDatabase creates a new database in the server.

func (*Server) GetMetrics added in v0.0.12 

func (s *Server) GetMetrics() map[string]string

func (*Server) Initialize 

func (s *Server) Initialize() error

Initialize is a method of the Server struct that initializes the server. It creates the server directory and loads the databases. The server directory is determined by the getDefaultServerDir function. If the server directory does not exist, it is created with read, write, and execute permissions for the user only. If there is an error creating the server directory, the error is returned. After the server directory is successfully created or if it already exists, the databases are loaded using the LoadDatabases method. If there is an error loading the databases, the error is returned. If the server directory is successfully created and the databases are successfully loaded, the method returns nil.

func (*Server) ListDatabases 

func (s *Server) ListDatabases() []string

ListDatabases returns a list of databases in the server.

func (*Server) LoadDatabases 

func (s *Server) LoadDatabases() error

LoadDatabases is a method of the Server struct that loads the databases from the server directory. It reads the server directory using the os.ReadDir function and the getDefaultServerDir function. If there is an error reading the server directory, the error is returned. For each directory in the server directory, it creates a new Database instance with the directory name as the database name. It then loads the tables from the database directory using the LoadTables method of the Database struct. If there is an error loading the tables, the error is returned. If the tables are successfully loaded, the database is added to the Databases field of the Server struct. If all databases are successfully loaded, the method returns nil.

func (*Server) RestoreDatabases added in v0.0.11 

func (s *Server) RestoreDatabases(backupPath ...string) error

RestoreDatabases is a method of the Server struct that restores databases from the latest backup file. It acquires a lock on the Server struct and defers the unlocking of the lock. It opens the latest backup file in the default backup directory. The default backup directory is determined by the getDefaultBackUpDir if the backup Path is empty, if there's not, the route will be checked. If there is an error opening the backup file, the error is returned. It gets the file stat of the backup file. If there is an error getting the file stat, the error is returned. It creates a new zip reader for the backup file. If there is an error creating the zip reader, the error is returned. It iterates over each file in the zip file. For each file, it creates the file path by joining the default server directory and the file name. The default server directory is determined by the getDefaultServerDir function. If the file is a directory, it creates the directory with read, write, and execute permissions for the user only. If the file is a regular file, it creates the file with the same permissions as in the zip file. It opens the file for writing. If there is an error opening the file, the error is returned. It opens the zip file for reading. If there is an error opening the zip file, the error is returned. It copies the contents of the zip file to the file. If there is an error copying the contents, the error is returned. It closes the file and the zip file. After all files are successfully restored, it loads the databases using the LoadDatabases method of the Server struct. If there is an error loading the databases, the error is returned. If all databases are successfully loaded, the method returns nil.

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface for the server.

type Table 

type Table struct {
	sync.RWMutex        // Mutex for read-write locking
	FilePath     string // Path to the file where the table data is stored
	PrimaryKey   string // Field name used as the primary key for the table

	Indexes   map[string][]*dbdata.Record // Map of field names to slices of records that have that field
	Records   map[string]*dbdata.Record   // Map of primary key values to the corresponding records
	Cache     map[string]*CacheEntry      // LRU cache for recently accessed records
	CacheSize int                         // Maximum cache size
	// contains filtered or unexported fields
}

func NewTable 

func NewTable(config TableConfig) (*Table, error)

NewTable creates a new Table instance with the given configuration. Returns an error instead of using log.Fatal for better error handling.

func NewTableSimple added in v0.0.22 

func NewTableSimple(primaryKey, filePath string) (*Table, error)

NewTableSimple creates a table with default configuration for backward compatibility

func (*Table) Count added in v0.0.22 

func (t *Table) Count(filters map[string]any) (int, error)

Count returns the number of records that match the given filters without retrieving them

func (*Table) Delete 

func (t *Table) Delete(key interface{}) error

Delete is a method of the Table struct that deletes a record from the table based on the given key. It locks the table for writing, ensuring that no other goroutines can modify the table while the deletion is happening. It first reads all existing records from the file where the table data is stored. If the primary key of the record to be deleted does not exist in the table, it returns an error. It then removes the record from the main records map. It also updates the indexes. For each field in the record, it removes the record from the index for that field. If the index for a field becomes empty after the removal of the record, it deletes the index. It then writes the updated records back to the file. If any error occurs during these operations, it returns the error.

Parameters: - key: An interface{} representing the key of the record to be deleted. The key is converted to a string before the deletion is performed.

Returns: - If the operation is successful, it returns nil. - If an error occurs, it returns the error.

func (*Table) DeleteMany added in v0.0.13 

func (t *Table) DeleteMany(keys []interface{}) []error

DeleteMany is a method of the Table struct that deletes multiple records from the table based on the given keys. It locks the table for writing, ensuring that no other goroutines can modify the table while the deletion is happening. It first reads all existing records from the file where the table data is stored. For each key, if the primary key of the record to be deleted does not exist in the table, it returns an error for that key but continues with the rest. It then removes the record from the main records map. It also updates the indexes. For each field in the record, it removes the record from the index for that field. If the index for a field becomes empty after the removal of the record, it deletes the index. It then writes the updated records back to the file. If any error occurs during these operations, it returns the error.

Parameters: - keys: A slice of interface{} representing the keys of the records to be deleted. The keys are converted to strings before the deletion is performed.

Returns: - A slice of errors for keys that failed to delete. If all records are deleted successfully, the slice is empty.

func (*Table) DeleteWithTransaction 

func (t *Table) DeleteWithTransaction(key any) error

DeleteWithTransaction performs a delete operation within a transaction context. It creates a transaction and uses ExecuteInTransaction for automatic rollback on error.

func (*Table) Insert 

func (t *Table) Insert(record Record) error

func (*Table) InsertMany added in v0.0.13 

func (t *Table) InsertMany(records []Record) error

InsertMany is a method of the Table struct that inserts multiple new records into the table. It divides the records into batches and inserts each batch separately to optimize performance. If an error occurs while inserting a batch, it is added to a slice of errors. After all batches have been processed, it returns the slice of errors.

Parameters: - records: A slice of maps representing the records to be inserted. The keys are field names and the values are the field values.

Returns: - A slice of errors for records that failed to insert. If all records are inserted successfully, the slice ismpty.

func (*Table) InsertWithTransaction 

func (t *Table) InsertWithTransaction(record Record) error

InsertWithTransaction performs an insert operation within a transaction context. It creates a transaction and uses ExecuteInTransaction for automatic rollback on error.

func (*Table) LoadIndexes 

func (t *Table) LoadIndexes() error

LoadIndexes loads the indexes from the file

func (*Table) NewTransaction added in v0.0.22 

func (t *Table) NewTransaction() *Transaction

NewTransaction creates a new transaction for this table. This allows external code to manage transactions manually if needed.

func (*Table) Query 

func (t *Table) Query(query Query) ([]Record, error)

Query executes a query on the table and returns matching records. Supports filtering, sorting, and pagination through the Query object.

func (*Table) QueryWithPlan added in v0.0.22 

func (t *Table) QueryWithPlan(query Query) ([]Record, ExecutionPlan, error)

QueryWithPlan executes a query and returns both results and execution plan for debugging

func (*Table) ResetAndLoadIndexes 

func (t *Table) ResetAndLoadIndexes() error

ResetAndLoadIndexes resets the indexes and reloads them from the file

func (*Table) Select 

func (t *Table) Select(key interface{}) (Record, error)

Select is a method of the Table struct that selects a record from the table based on the given key. It locks the table for reading, ensuring that no other goroutines can modify the table while the selection is happening. It then reads all existing records from the file where the table data is stored. It converts the key to a string and checks if a record with that key exists in the table. If a record with that key does not exist, it returns an error and a nil record. If a record with that key exists, it returns the record and a nil error.

Parameters: - key: An interface{} representing the key of the record to be selected. The key is converted to a string before the selection is performed.

Returns: - A pointer to a dbdata.Record instance representing the record with the given key. - If a record with the given key does not exist, it returns an error and a nil record. - If an error occurs while reading the records from the file, it returns the error and a nil record. - If the operation is successful, it returns the record with the given key and a nil error.

func (*Table) SelectAll 

func (t *Table) SelectAll() ([]Record, error)

SelectAll is a method of the Table struct that selects all records from the table. It locks the table for reading, ensuring that no other goroutines can modify the table while the selection is happening. It then reads all existing records from the file where the table data is stored. It iterates over the records, appending each one to a slice of records. If any error occurs during these operations, it returns the error and a nil slice. If the operation is successful, it returns the slice of all records and a nil error.

Returns: - A slice of pointers to dbdata.Record instances representing all records in the table. - If an error occurs, it returns the error and a nil slice. - If the operation is successful, it returns the slice of all records and a nil error.

func (*Table) SelectWithFilter 

func (t *Table) SelectWithFilter(filters map[string]interface{}) ([]Record, error)

SelectWithFilter is a method of the Table struct that selects records from the table based on the given filters. It locks the table for reading, ensuring that no other goroutines can modify the table while the selection is happening. It then reads all existing records from the file where the table data is stored. It iterates over the records, checking each one against the filters. For each record, it iterates over the filters. For each filter, it converts the filter value to a proto Value. If an error occurs during this conversion, it returns the error and a nil slice. It then checks if the field specified by the filter exists in the record and if the value of the field in the record is equal to the filter value. If the field does not exist in the record or if the values are not equal, it skips to the next record. If all filters match for a record, it appends the record to a slice of matched records. If the operation is successful, it returns the slice of matched records and a nil error.

Parameters: - filters: A map where the keys are field names and the values are the filter values. Only records where the field values match the filter values are selected.

Returns: - A slice of pointers to dbdata.Record instances representing the records that match the filters. - If an error occurs, it returns the error and a nil slice. - If the operation is successful, it returns the slice of matched records and a nil error.

func (*Table) Update 

func (t *Table) Update(key interface{}, updates Record) error

Update is a method of the Table struct that updates a record in the table based on the given key. It locks the table for writing, ensuring that no other goroutines can modify the table while the update is happening. It first reads all existing records from the file where the table data is stored. If the primary key of the record to be updated does not exist in the table, it returns an error. It then iterates over the fields in the updates map, updating each field in the existing record. For each field, it checks if the field exists in the existing record. If the field exists, it removes the existing record from the index for that field. It then converts the new field value to a proto Value and updates the field in the existing record. If an error occurs during this conversion, it returns the error. It then adds the updated record to the index for the field. If the index for the field does not exist, it initializes it before adding the updated record. It then writes the updated records back to the file. If any error occurs during these operations, it returns the error.

Parameters: - key: An interface{} representing the key of the record to be updated. The key is converted to a string before the update is performed. - updates: A map representing the fields to be updated in the record. The keys are field names and the values are the new field values.

Returns: - If the operation is successful, it returns nil. - If an error occurs, it returns the error.

func (*Table) UpdateMany added in v0.0.13 

func (t *Table) UpdateMany(updates map[string]Record) []error

UpdateMany is a method of the Table struct that updates multiple records in the table based on the given keys and updates. It locks the table for writing, ensuring that no other goroutines can modify the table while the updates are happening. It first reads all existing records from the file where the table data is stored. For each key, if the primary key of the record to be updated does not exist in the table, it returns an error for that key but continues with the rest. It then iterates over the fields in the updates map, updating each field in the existing record. For each field, it checks if the field exists in the existing record. If the field exists, it removes the existing record from the index for that field. It then converts the new field value to a proto Value and updates the field in the existing record. If an error occurs during this conversion, it returns an error for that record but continues with the rest. It then adds the updated record to the index for the field. If the index for the field does not exist, it initializes it before adding the updated record. It then writes the updated records back to the file. If any error occurs during these operations, it returns an error for that record but continues with the rest.

Parameters: - updates: A map where the keys are the primary key values and the values are the updates to be applied to the records. Each value is a map representing the fields to be updated.

Returns: - A slice of errors for records that failed to update. If all records are updated successfully, the slice is empty.

func (*Table) UpdateWithTransaction 

func (t *Table) UpdateWithTransaction(key any, updates Record) error

UpdateWithTransaction performs an update operation within a transaction context. It creates a transaction and uses ExecuteInTransaction for automatic rollback on error.

type TableConfig added in v0.0.22 

type TableConfig struct {
	PrimaryKey    string
	FilePath      string
	CacheSize     int  // Maximum number of records to cache
	EnableMetrics bool // Whether to enable metrics collection
}

TableConfig holds configuration options for creating a new table

type TableMetadata added in v0.0.22 

type TableMetadata struct {
	PrimaryKey string `json:"PrimaryKey"`
}

type Transaction 

type Transaction struct {
	sync.Mutex                                // Mutex to ensure the transaction is thread safe
	OriginalRecords map[string]*dbdata.Record // Map to hold the original records
	Table           *Table                    // Table to which the transaction belongs
}

This srtuct holds the transaction data for managing the transaction

func NewTransaction 

func NewTransaction(table *Table) *Transaction

Creates a new transaction with the table

func (*Transaction) Commit 

func (t *Transaction) Commit() error

Commit ends the transaction by unlocking it, indicating successful completion of all operations

func (*Transaction) Delete added in v0.0.22 

func (t *Transaction) Delete(key interface{}) error

Delete performs a delete operation within this transaction

func (*Transaction) DeleteMany added in v0.0.22 

func (t *Transaction) DeleteMany(keys []interface{}) []error

DeleteMany performs multiple delete operations within this transaction

func (*Transaction) ExecuteInTransaction added in v0.0.22 

func (t *Transaction) ExecuteInTransaction(fn func(*Transaction) error) error

ExecuteInTransaction executes a function within a transaction context If the function returns an error, the transaction is rolled back If the function succeeds, the transaction is committed

func (*Transaction) Insert added in v0.0.22 

func (t *Transaction) Insert(record Record) error

Insert performs an insert operation within this transaction

func (*Transaction) InsertMany added in v0.0.22 

func (t *Transaction) InsertMany(records []Record) error

InsertMany performs multiple insert operations within this transaction

func (*Transaction) Rollback 

func (t *Transaction) Rollback() error

Rollback ends the transaction by unlocking it and writing the original records back to the file

func (*Transaction) Start 

func (t *Transaction) Start() error

Start begins the transaction by locking the table and duplicating the records for potential rollback

func (*Transaction) Update added in v0.0.22 

func (t *Transaction) Update(key interface{}, updates Record) error

Update performs an update operation within this transaction

func (*Transaction) UpdateMany added in v0.0.22 

func (t *Transaction) UpdateMany(updates map[string]Record) []error

UpdateMany performs multiple update operations within this transaction

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.