onlineddl: support regular expressions rules for PT/GH-OST #2139
Conversation
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. DetailsReviewer can indicate their review by submitting an approval review. |
|
Please fix the integration test error on CI as well 🤣 This is because you have changed the config of the task and can be fixed by referring to this commit Note that in your case you should not use |
| ShadowTableRule []string `yaml:"shadow-table-rule" toml:"shadow-table-rule" json:"shadow-table-rule"` | ||
| TrashTableRule []string `yaml:"trash-table-rule" toml:"trash-table-rule" json:"trash-table-rule"` |
There was a problem hiding this comment.
| ShadowTableRule []string `yaml:"shadow-table-rule" toml:"shadow-table-rule" json:"shadow-table-rule"` | |
| TrashTableRule []string `yaml:"trash-table-rule" toml:"trash-table-rule" json:"trash-table-rule"` | |
| ShadowTableRules []string `yaml:"shadow-table-rules" toml:"shadow-table-rules" json:"shadow-table-rules"` | |
| TrashTableRules []string `yaml:"trash-table-rules" toml:"trash-table-rules" json:"trash-table-rules"` |
|
|
||
| OnlineDDL bool `yaml:"online-ddl" toml:"online-ddl" json:"online-ddl"` | ||
| // pt/gh-ost name rule,support regex | ||
| ShadowTableRule []string `yaml:"shadow-table-rule" toml:"shadow-table-rule" json:"shadow-table-rule"` |
| ErrConfigGenTableRouter = New(codeConfigGenTableRouter, ClassConfig, ScopeInternal, LevelHigh, "generate table router error", "Please check the `routes` config in task configuration file.") | ||
| ErrConfigGenColumnMapping = New(codeConfigGenColumnMapping, ClassConfig, ScopeInternal, LevelHigh, "generate column mapping error", "Please check the `column-mappings` config in task configuration file.") | ||
| ErrConfigInvalidChunkFileSize = New(codeConfigInvalidChunkFileSize, ClassConfig, ScopeInternal, LevelHigh, "invalid `chunk-filesize` %v", "Please check the `chunk-filesize` config in task configuration file.") | ||
| ErrConfigOnlineDDLRegexCompile = New(codeConfigOnlineDDLRegexCompile, ClassConfig, ScopeInternal, LevelHigh, |
There was a problem hiding this comment.
I think we can merge this two error code together, the only reason that regexp compile fail is that the syntax is not correct.
| adjustedRules := make([]string, 0, len(rules)) | ||
| for _, r := range rules { | ||
| if !strings.HasPrefix(r, "^") { | ||
| r = "^" + r |
There was a problem hiding this comment.
perhaps it would be better to just return the error here rather than adjust re ((add "^" and "$") ) for the user
There was a problem hiding this comment.
I think this way is simpler for users. They need to check the error message and rewrite task.yaml if we return an error here. Waiting for other reviewers' suggestions...
|
/run-unit-tests |
|
/merge |
|
This pull request has been accepted and is ready to merge. DetailsCommit hash: 5850fb4 |
|
In response to a cherrypick label: new pull request created: #2156. |
What problem does this PR solve?
Solve the problem of custom PT/GH-OST format
This feature is added because some users may modify the pt/gh-ost code because they are worried about the business mistakenly deleting the table, which may cause the format to be inconsistent with the Dm, which affects synchronization，the feature can solve it
close #2123
What is changed and how it works?
Add DM config，if values is null, dm will keep default , if values is not null , match according to specific value
shadow-table-rule: [] # 支持正则，默认
^_(.+)_(?:new|gho)$匹配多个具体值后加(.*)trash-table-rule: [] # 支持正则, 默认
^_(.+)_(?:ghc|del|old)$匹配多个具体值后加(.*)