|
2019-06-17
§
|
| 14:32 |
<otto@deploy1001> |
scap-helm eventgate-main cluster codfw completed |
[production] |
| 14:32 |
<otto@deploy1001> |
scap-helm eventgate-main upgrade main -f /srv/scap-helm/eventgate/main/codfw-values.yaml --reset-values stable/eventgate [namespace: eventgate-main, clusters: codfw] |
[production] |
| 14:26 |
<otto@deploy1001> |
scap-helm eventgate-main finished |
[production] |
| 14:26 |
<otto@deploy1001> |
scap-helm eventgate-main cluster staging completed |
[production] |
| 14:26 |
<otto@deploy1001> |
scap-helm eventgate-main upgrade main -f /srv/scap-helm/eventgate/main/staging-values.yaml --reset-values stable/eventgate [namespace: eventgate-main, clusters: staging] |
[production] |
| 14:15 |
<moritzm> |
installing poppler security updates on jessie |
[production] |
| 14:03 |
<cdanis> |
cdanis@cobalt.wikimedia.org ~ % sudo systemctl start gerrit.service |
[production] |
| 13:53 |
<ema@cumin1001> |
END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) |
[production] |
| 13:49 |
<moritzm> |
installing libav security updates |
[production] |
| 13:45 |
<ema@cumin1001> |
START - Cookbook sre.hosts.upgrade-and-reboot |
[production] |
| 13:45 |
<ema> |
reboot cp4027 for dist and Varnish upgrade T224694 |
[production] |
| 13:34 |
<elukey> |
reboot of an-worker* (Hadoop worker nodes) for kernel + openjdk upgrades |
[production] |
| 13:25 |
<ema> |
cp4027: upgrade Varnish packages to 5.1.3-1wm10 T224694 |
[production] |
| 12:37 |
<jbond42> |
upgrade mtail on lithium - T225604 |
[production] |
| 12:35 |
<jbond42> |
add mtail_3.0.0~rc24.1-1+wmf1_amd64.deb to jessie-wikimedia backports |
[production] |
| 12:13 |
<awight@deploy1001> |
Synchronized wmf-config/CommonSettings-labs.php: SWAT: [[gerrit:514715|FileImporter configuration to fetch sitelinks from Wikidata (T225609 T224007)]] - finishing partial deployment (duration: 00m 47s) |
[production] |
| 12:06 |
<awight> |
EU SWAT complete |
[production] |
| 12:05 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|517391 Enable AMC mode for Persian, Japanese, Thai and Italian wikis (T225123)]] (duration: 00m 47s) |
[production] |
| 12:02 |
<Urbanecm> |
EU SWAT is going a few minutes beyond its window |
[production] |
| 11:55 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|516608 Enable feature flag for breaking Wikibase API change (T223303)]] (duration: 00m 47s) |
[production] |
| 11:49 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|516478 Set EntityUsageTable addUsage batch size to 200 (T225500)]] (duration: 00m 47s) |
[production] |
| 11:46 |
<awight@deploy1001> |
Synchronized php-1.34.0-wmf.8/extensions/ContentTranslation: SWAT: [[gerrit:514734|Fix undefined index notices (T225198)]] (duration: 00m 49s) |
[production] |
| 11:33 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:517371|Add autoreview protection level on ar.wikipedia (T225896)]] (duration: 00m 47s) |
[production] |
| 11:28 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:512220|Enable VisualEditor in draft namespace on sr.wiki (T223024)]] (duration: 00m 47s) |
[production] |
| 11:23 |
<awight> |
ran mwscript namespaceDupes.php nds_nlwiki, no dupes found |
[production] |
| 11:22 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:517372|Set nds_nlwiki's sitename and metanamespace back to defaults (T224349)]] (duration: 00m 47s) |
[production] |
| 11:11 |
<awight@deploy1001> |
Synchronized wmf-config/CommonSettings.php: wmf-config/CommonSettings-labs.php SWAT: [[gerrit:514715|FileImporter configuration to fetch sitelinks from Wikidata (T225609 T224007)]] (duration: 00m 47s) |
[production] |
| 10:51 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:517404| Bumping portals to master (T128546)]] (duration: 00m 47s) |
[production] |
| 10:51 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:517404| Bumping portals to master (T128546)]] (duration: 00m 49s) |
[production] |
| 09:39 |
<moritzm> |
rebooting mw2184, mw1265 for some tests |
[production] |
| 09:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:36 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:36 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:31 |
<elukey> |
set cpu governor to performance (was powersave) on analytics1070 (hadoop worker node) |
[production] |
| 09:17 |
<moritzm> |
rebooting sulfur for some tests |
[production] |
| 09:15 |
<_joe_> |
The governor was set to "powersave", not "ondemand" |
[production] |
| 09:13 |
<_joe_> |
setting cpufreq governor to "ondemand" on mw1348, T225713 |
[production] |
| 08:52 |
<onimisionipe> |
remove maps1001 from cassandra cluster - T224395 |
[production] |
| 07:25 |
<XioNoX> |
restart snmp daemon on mr1-eqsin |
[production] |
| 07:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2107 (duration: 00m 47s) |
[production] |
| 06:22 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2084 (duration: 00m 47s) |
[production] |
| 06:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2084 for a reboot (duration: 00m 48s) |
[production] |
| 06:04 |
<marostegui> |
Stop MySQ on db2084 to reboot the host T225884 |
[production] |
| 05:16 |
<marostegui> |
Stop MySQL on db2107 to clone db2051 - T221533 |
[production] |
| 05:11 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2107 to clone db2051 (duration: 00m 47s) |
[production] |
| 05:03 |
<marostegui> |
Optimize all pc1008's tables T210725 |
[production] |
| 05:03 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool pc1008 and pool pc1010 temporarily while pc1008 gets all its tables optimized T210725 (duration: 00m 59s) |
[production] |