1601-1650 of 10000 results (41ms)
2021-05-26 §
14:31 <moritzm> updated bullseye d-i image to 2021-05-26 daily image T275873 [production]
14:19 <otto@deploy1002> Started deploy [analytics/refinery@b787999]: Regular analytics weekly train [analytics/refinery@e536abd] [production]
14:18 <otto@deploy1002> deploy aborted: Regular analytics weekly train [analytics/refinery@e536abd] (duration: 00m 06s) [production]
14:18 <otto@deploy1002> Started deploy [analytics/refinery@e536abd]: Regular analytics weekly train [analytics/refinery@e536abd] [production]
14:05 <mbsantos@deploy1002> Finished deploy [kartotherian/deploy@5d7c993]: (no justification provided) (duration: 00m 14s) [production]
14:05 <mbsantos@deploy1002> Started deploy [kartotherian/deploy@5d7c993]: (no justification provided) [production]
14:03 <hashar@deploy1002> Finished deploy [integration/docroot@ebee5d3]: composer/npm updates (duration: 00m 09s) [production]
14:03 <hashar@deploy1002> Started deploy [integration/docroot@ebee5d3]: composer/npm updates [production]
11:44 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.6/extensions/GrowthExperiments/maintenance/fixLinkRecommendationData.php: b3c2941: Allow running fixLinkRecommendationData --search-index in production (T283606) (duration: 01m 07s) [production]
11:39 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.7/extensions/GrowthExperiments/maintenance/fixLinkRecommendationData.php: 86bba48: Allow running fixLinkRecommendationData --search-index in production (T283606) (duration: 01m 06s) [production]
11:30 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
11:30 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
11:27 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.7/extensions/GrowthExperiments/: GrowthExperiments backports (T283544; T282899; T282546) (duration: 01m 06s) [production]
11:26 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.6/extensions/GrowthExperiments/: GrowthExperiments backports (T283544; T282899; T282546) (duration: 01m 19s) [production]
11:05 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:694339|Test Wikidata: Enable empty list to object serialization (T241422)]] (duration: 01m 19s) [production]
10:26 <moritzm> installing lz4 security updates on buster [production]
10:01 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 180 days, 0:00:00 on labstore1007.wikimedia.org with reason: T281045 [production]
10:01 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime for 180 days, 0:00:00 on labstore1007.wikimedia.org with reason: T281045 [production]
09:55 <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.6/extensions/Wikibase: Backport: [[gerrit:695037|Wrap list of acceptable site ids with an APCu cache in API]] (duration: 01m 18s) [production]
09:45 <godog> rm /root/prometheus from prometheus5001 - old transition files [production]
09:42 <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.7/extensions/Wikibase: Backport: [[gerrit:695035|Wrap list of acceptable site ids with an APCu cache in API]] (duration: 02m 12s) [production]
09:36 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 100%: Repool db1106', diff saved to https://phabricator.wikimedia.org/P16222 and previous config saved to /var/cache/conftool/dbconfig/20210526-093647-root.json [production]
09:21 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 75%: Repool db1106', diff saved to https://phabricator.wikimedia.org/P16221 and previous config saved to /var/cache/conftool/dbconfig/20210526-092144-root.json [production]
09:13 <elukey> deploy https://gerrit.wikimedia.org/r/c/operations/homer/public/+/695192 on {cr1|cr2}-eqiad - T225005 [production]
09:06 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 50%: Repool db1106', diff saved to https://phabricator.wikimedia.org/P16220 and previous config saved to /var/cache/conftool/dbconfig/20210526-090640-root.json [production]
08:51 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 25%: Repool db1106', diff saved to https://phabricator.wikimedia.org/P16219 and previous config saved to /var/cache/conftool/dbconfig/20210526-085137-root.json [production]
08:12 <_joe_> purging images on deneb [production]
08:11 <kormat> running 'optimize table' over parsercache db on pc1007 with replication enabled T282761 [production]
07:14 <ryankemper> Pooled `wdqs1013` (caught up on lag), de-pooled `wdqs2003` (should not have been pooled due to reimage failure) [production]
07:13 <ryankemper@puppetmaster2001> conftool action : set/pooled=no; selector: name=wdqs2003.codfw.wmnet [production]
05:46 <marostegui> Stop MySQL on clouddb1021 to upgrade mysql [production]
05:19 <marostegui@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 100%: Repool db1160', diff saved to https://phabricator.wikimedia.org/P16215 and previous config saved to /var/cache/conftool/dbconfig/20210526-051935-root.json [production]
05:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1148', diff saved to https://phabricator.wikimedia.org/P16214 and previous config saved to /var/cache/conftool/dbconfig/20210526-050919-marostegui.json [production]
05:04 <marostegui@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 75%: Repool db1160', diff saved to https://phabricator.wikimedia.org/P16213 and previous config saved to /var/cache/conftool/dbconfig/20210526-050431-root.json [production]
04:49 <marostegui@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 50%: Repool db1160', diff saved to https://phabricator.wikimedia.org/P16212 and previous config saved to /var/cache/conftool/dbconfig/20210526-044928-root.json [production]
04:35 <marostegui> Deploy schema change on db1106, this will generate lag on s1 (enwiki) on wiki replicas T266486 T268392 T273360 [production]
04:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1106', diff saved to https://phabricator.wikimedia.org/P16211 and previous config saved to /var/cache/conftool/dbconfig/20210526-043439-marostegui.json [production]
04:34 <marostegui@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 25%: Repool db1160', diff saved to https://phabricator.wikimedia.org/P16210 and previous config saved to /var/cache/conftool/dbconfig/20210526-043424-root.json [production]
03:29 <eileen> process-control config revision is 7b646533da [production]
00:47 <eileen> civicrm revision changed from 584b96452a to eac772e9c9, config revision is 2ca92c3c3c [production]
00:27 <mutante> phab2001 - restarted apache2 [production]
2021-05-25 §
23:09 <razzi@cumin1001> END (PASS) - Cookbook sre.hadoop.roll-restart-masters (exit_code=0) [production]
22:39 <razzi@cumin1001> START - Cookbook sre.hadoop.roll-restart-masters [production]
22:21 <razzi@cumin1001> END (FAIL) - Cookbook sre.hadoop.roll-restart-masters (exit_code=99) [production]
22:21 <razzi@cumin1001> START - Cookbook sre.hadoop.roll-restart-masters [production]
22:21 <razzi@cumin1001> END (FAIL) - Cookbook sre.hadoop.roll-restart-masters (exit_code=99) [production]
22:21 <razzi@cumin1001> START - Cookbook sre.hadoop.roll-restart-masters [production]
22:04 <razzi@cumin1001> END (FAIL) - Cookbook sre.hadoop.roll-restart-masters (exit_code=99) [production]
22:04 <razzi@cumin1001> START - Cookbook sre.hadoop.roll-restart-masters [production]
21:58 <razzi@cumin1001> END (FAIL) - Cookbook sre.hadoop.roll-restart-masters (exit_code=99) [production]