Skip to content

Conversation

@shlomi-noach
Copy link
Contributor

Fixes #418

This PR introduces --critical-load-hibernate-seconds. This flag defaults 0 (in which case the new feature introduced in this PR is disabled)

when critical-load-hibernate-seconds > 0, then the behavior of critical-load changes. Instead of panicking and bailing out with error, gh-ost enters hibernation.

As opposed to throttling, hibernation implies absolutely no queries on neither master not migrated server. No heartbeats; no throttle checks; no replication lag checks; nothing.

When gh-ost awakes from hibernation, it's all back to normal, and throttling checks take place as usual, and of course it may potentially re-enter hibernation if critical-load is still met.

cc @ggunson @github/database-infrastructure

@shlomi-noach
Copy link
Contributor Author

TODO: on leaving hibernation, gh-ost has no up-to-date metrics. It shouldn't just resume work; it must first run a collection of throttle-metrics, inject heartbeat etc. Only then should it resume work.

This is somewhat complicated to achieve and supervise, since most of these processes are independent.

@shlomi-noach shlomi-noach deployed to production/ghost-mysql001 May 24, 2017 06:06 Active
@shlomi-noach
Copy link
Contributor Author

@shlomi-noach shlomi-noach deployed to production/ghost-mysql001 May 24, 2017 07:54 Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants