Support engine options and refactor github engine

[junnplus]

I hereby agree to the terms of the CLA available at: https://databend.rs/policies/cla/

Summary

• Add engine_options field to DatabaseMeta and TableMeta
• Change table_engine_github_enabled config flag to database_engine_github_enabled
• Use GITHUB engine to replace GITHUB_REPO_XXX_TABLE_ENGINE table engine

mysql> create database datafuselabs engine=github(token='xxx');
Query OK, 0 rows affected (1.21 sec)
Read 0 rows, 0 B in 1.203 sec., 0 rows/sec., 0 B/sec.

mysql> use datafuselabs;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-------------------------------+---------------------------------+
| created_on                    | name                            |
+-------------------------------+---------------------------------+
| 2021-12-19 11:20:37.844 +0000 | .github                         |
| 2021-12-19 11:20:37.847 +0000 | .github_comments                |
| 2021-12-19 11:20:37.845 +0000 | .github_issues                  |
| 2021-12-19 11:20:37.846 +0000 | .github_prs                     |
| 2021-12-19 11:20:37.806 +0000 | databend                        |
| 2021-12-19 11:20:37.836 +0000 | databend-playground             |
| 2021-12-19 11:20:37.839 +0000 | databend-playground_comments    |
| 2021-12-19 11:20:37.837 +0000 | databend-playground_issues      |
| 2021-12-19 11:20:37.838 +0000 | databend-playground_prs         |
| 2021-12-19 11:20:37.814 +0000 | databend_comments               |
| 2021-12-19 11:20:37.810 +0000 | databend_issues                 |
| 2021-12-19 11:20:37.813 +0000 | databend_prs                    |
| 2021-12-19 11:20:37.852 +0000 | datafuse-operator               |
| 2021-12-19 11:20:37.855 +0000 | datafuse-operator_comments      |
| 2021-12-19 11:20:37.853 +0000 | datafuse-operator_issues        |
| 2021-12-19 11:20:37.854 +0000 | datafuse-operator_prs           |
| 2021-12-19 11:20:37.826 +0000 | datafuse-presentations          |
| 2021-12-19 11:20:37.834 +0000 | datafuse-presentations_comments |
| 2021-12-19 11:20:37.828 +0000 | datafuse-presentations_issues   |
| 2021-12-19 11:20:37.829 +0000 | datafuse-presentations_prs      |
| 2021-12-19 11:20:37.821 +0000 | datafuse-shop                   |
| 2021-12-19 11:20:37.825 +0000 | datafuse-shop_comments          |
| 2021-12-19 11:20:37.822 +0000 | datafuse-shop_issues            |
| 2021-12-19 11:20:37.823 +0000 | datafuse-shop_prs               |
| 2021-12-19 11:20:37.840 +0000 | fusebots                        |
| 2021-12-19 11:20:37.843 +0000 | fusebots_comments               |
| 2021-12-19 11:20:37.841 +0000 | fusebots_issues                 |
| 2021-12-19 11:20:37.842 +0000 | fusebots_prs                    |
| 2021-12-19 11:20:37.848 +0000 | test-infra                      |
| 2021-12-19 11:20:37.851 +0000 | test-infra_comments             |
| 2021-12-19 11:20:37.849 +0000 | test-infra_issues               |
| 2021-12-19 11:20:37.850 +0000 | test-infra_prs                  |
| 2021-12-19 11:20:37.816 +0000 | weekly                          |
| 2021-12-19 11:20:37.820 +0000 | weekly_comments                 |
| 2021-12-19 11:20:37.817 +0000 | weekly_issues                   |
| 2021-12-19 11:20:37.818 +0000 | weekly_prs                      |
+-------------------------------+---------------------------------+
36 rows in set (0.01 sec)
Read 52 rows, 4.99 KB in 0.007 sec., 7.83 thousand rows/sec., 752.15 KB/sec.

mysql> show create table databend_issues;
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table           | Create Table                                                                                                                                                                                                                                                          |
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| databend_issues | CREATE TABLE `databend_issues` (
  `number` Int64,
  `title` String,
  `state` String,
  `user` String,
  `labels` String,
  `assigness` String,
  `comments` UInt32,
  `created_at` DateTime32,
  `updated_at` DateTime32,
  `closed_at` DateTime32,
) ENGINE=GITHUB |
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)
Read 0 rows, 0 B in 0.004 sec., 0 rows/sec., 0 B/sec.

todo:

• make github database immutable

Changelog

• New Feature

Related Issues

Fixes #issue

Test Plan

Unit Tests

Stateless Tests

[databend-bot]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[codecov-commenter]

Codecov Report

Merging #3542 (fd40fdf) into main (ffafbf1) will decrease coverage by 0%.
The diff coverage is 21%.

@@          Coverage Diff           @@
##            main   #3542    +/-   ##
======================================
- Coverage     60%     60%    -1%     
======================================
  Files        624     630     +6     
  Lines      34787   34923   +136     
======================================
+ Hits       21203   21285    +82     
- Misses     13584   13638    +54     

Impacted Files	Coverage Δ
common/meta/raft-store/src/state_machine/sm.rs	97% <ø> (ø)
query/src/databases/default/default_database.rs	77% <ø> (ø)
query/src/databases/github/github_database.rs	0% <0%> (ø)
query/src/storages/github/github_client.rs	0% <0%> (ø)
query/src/storages/github/github_table.rs	0% <0%> (ø)
query/src/storages/github/repo.rs	0% <0%> (ø)
query/src/storages/github/repo_comments_table.rs	0% <0%> (ø)
query/src/storages/github/repo_info_table.rs	0% <0%> (ø)
query/src/storages/github/repo_issues_table.rs	0% <0%> (ø)
query/src/storages/github/repo_prs_table.rs	0% <0%> (ø)
... and 44 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffafbf1...fd40fdf. Read the comment docs.

[Veeupup]

Great Abstraction!

[flaneur2020]

ls +1!

[Veeupup]

/LGTM

[databend-bot]

Wait for another reviewer approval

[databend-bot]

CI Passed
Reviewers Approved
Let's Merge
Thank you for the PR @junnplus

[BohuTANG]

Looks great, thank you @junnplus !