Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain,infoschema: avoid v1/v2 switch when loading snapshot infoschema #55129

Merged
merged 6 commits into from
Aug 3, 2024

Conversation

tiancaiamao
Copy link
Contributor

@tiancaiamao tiancaiamao commented Aug 1, 2024

What problem does this PR solve?

Issue Number: close #55114

Problem Summary:

What changed and how does it work?

Compatibility between infoschema snapshot and v1&v2 switch is difficult.
snapshot needs the history, while v1&v2 swtich reset the infoschema history,
this conflict is the root cause of bugs like #55114 #54926

So what I do in this commit is to avoid such case happen together:
If it's a loading of snapshot infoschema, then avoid v1&v2 switch.

Now whether the infoschema builder should build v2 is not decided soly by variable.SchemaCacheSize > 0, it's also affect by whether this loading isSnapshot. Provide a shouldUseV2() function for that.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

I try to add a unit test like this:

func TestIssue55114(t *testing.T) {
	store, dom := testkit.CreateMockStoreAndDomain(t)
	tk := testkit.NewTestKit(t, store)
	tk2 := testkit.NewTestKit(t, store)
	tk.MustExec("use test")
	// For mocktikv, safe point is not initialized, we manually insert it for snapshot to use.
	safePointName := "tikv_gc_safe_point"
	safePointValue := "20160102-15:04:05 -0700"
	safePointComment := "All versions after safe point can be accessed. (DO NOT EDIT)"
	updateSafePoint := fmt.Sprintf(`INSERT INTO mysql.tidb VALUES ('%[1]s', '%[2]s', '%[3]s')
	ON DUPLICATE KEY
	UPDATE variable_value = '%[2]s', comment = '%[3]s'`, safePointName, safePointValue, safePointComment)
	tk.MustExec(updateSafePoint)

	time1 := time.Now()
	// time1TS := oracle.GoTimeToTS(time1)
	schemaVer1 := tk.Session().GetInfoSchema().SchemaMetaVersion()
	time.Sleep(50*time.Millisecond)
	tk.MustExec("create table t (id int)")
	fmt.Println("before .. version ==", schemaVer1)

	go func() {
		for {
			tk.MustExec("set @@global.tidb_schema_cache_size = 0")
			dom.Reload()
			tk.MustExec("set @@global.tidb_schema_cache_size = 512 * 1024 * 1024")
			dom.Reload()
		}
	}()

	go func() {
		for {
			tk2.MustExec(fmt.Sprintf(`set @@tidb_snapshot="%s"`, time1.Format("2006-1-2 15:04:05.000")))
			time.Sleep(10*time.Millisecond)
		}
	}()

	for i:=0; i<1000; i++ {
		time.Sleep(100 * time.Millisecond)
		v := dom.InfoSchema().SchemaMetaVersion()
		fmt.Println("v ===", v)
		require.NotEqual(t, v, schemaVer1)
	}
}

The test works well and prove the correctness of this change.
But it's not feasible to be added to unit test, because how long it runs is undetermined.

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 1, 2024
Copy link

tiprow bot commented Aug 1, 2024

Hi @tiancaiamao. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

codecov bot commented Aug 1, 2024

Codecov Report

Attention: Patch coverage is 73.33333% with 8 lines in your changes missing coverage. Please review.

Project coverage is 74.3777%. Comparing base (52303d4) to head (b255561).
Report is 1 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #55129        +/-   ##
================================================
+ Coverage   72.9177%   74.3777%   +1.4599%     
================================================
  Files          1568       1568                
  Lines        439181     439192        +11     
================================================
+ Hits         320241     326661      +6420     
+ Misses        99308      92133      -7175     
- Partials      19632      20398       +766     
Flag Coverage Δ
integration 20.6282% <73.3333%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.9567% <ø> (ø)
parser ∅ <ø> (∅)
br 61.0830% <ø> (+15.1897%) ⬆️

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Aug 1, 2024
Copy link

ti-chi-bot bot commented Aug 1, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lance6716, wjhuang2016

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 1, 2024
Copy link

ti-chi-bot bot commented Aug 1, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-08-01 08:58:04.474545499 +0000 UTC m=+432600.754593568: ☑️ agreed by wjhuang2016.
  • 2024-08-01 09:07:43.729541389 +0000 UTC m=+433180.009589458: ☑️ agreed by lance6716.

@tiancaiamao
Copy link
Contributor Author

/retest

Copy link

tiprow bot commented Aug 1, 2024

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tiancaiamao
Copy link
Contributor Author

/ok-to-test

@ti-chi-bot ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Aug 1, 2024
@lance6716
Copy link
Contributor

/retest

@lance6716
Copy link
Contributor

/check-issue-triage-complete

@hawkingrei
Copy link
Member

/retest

@tiancaiamao
Copy link
Contributor Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit 52b4c8a into pingcap:master Aug 3, 2024
24 checks passed
@tiancaiamao tiancaiamao deleted the issue55114 branch August 3, 2024 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

domain information schema may be wrong if switching schema cache v1/v2 and getting snapshot information schema
4 participants