551-600 of 10000 results (54ms)
2019-09-11 §
10:57 <mobrovac> drop the wiktionary definition keyspace - T231361 [production]
10:23 <moritzm> removed roentgenium/tureis in Ganeti T224559 [production]
10:18 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
10:18 <jmm@cumin2001> START - Cookbook sre.hosts.decommission [production]
10:17 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
10:17 <jmm@cumin2001> START - Cookbook sre.hosts.decommission [production]
10:01 <jynus> stopping and upgrading db1074 [production]
09:56 <jynus> upgrading mariadb client libary on mariadb root clients [production]
09:46 <jiji@deploy1001> Synchronized wmf-config/CommonSettings.php: Push PHP7 traffic to 50% - T219150 (duration: 01m 03s) [production]
09:45 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #3a (duration: 12m 15s) [production]
09:32 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #3a [production]
09:32 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #3 (duration: 13m 18s) [production]
09:19 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #3 [production]
09:16 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #2 (duration: 03m 59s) [production]
09:13 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints, take #2 [production]
09:11 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints - T231361 T232449 (duration: 03m 24s) [production]
09:08 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf2ca76]: Stop using storage for enwiktionary definition and expose new PCS javascript endpoints - T231361 T232449 [production]
08:36 <mobrovac@deploy1001> Finished deploy [changeprop/deploy@7a8ab89]: Stop pregenerating enwiktionary page/definition, take #2 - T231361 (duration: 02m 13s) [production]
08:34 <mobrovac@deploy1001> Started deploy [changeprop/deploy@7a8ab89]: Stop pregenerating enwiktionary page/definition, take #2 - T231361 [production]
08:24 <mobrovac@deploy1001> Finished deploy [changeprop/deploy@069d297]: Revert Stop pregenerating enwiktionary page/definition (duration: 00m 34s) [production]
08:24 <mobrovac@deploy1001> Started deploy [changeprop/deploy@069d297]: Revert Stop pregenerating enwiktionary page/definition [production]
08:22 <mobrovac@deploy1001> Finished deploy [changeprop/deploy@56a8342]: Stop pregenerating enwiktionary page/definition - T231361 (duration: 02m 45s) [production]
08:19 <mobrovac@deploy1001> Started deploy [changeprop/deploy@56a8342]: Stop pregenerating enwiktionary page/definition - T231361 [production]
08:13 <elukey> add thirdparty/amd-rocm271 to buster-wikimedia and update it with ROCm 2.7.1 packages [production]
08:09 <jmm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:07 <elukey> execute reprepro clearvanished on install1002 to clear buster-wikimedia|thirdparty/amd-rocm27 (not used anymore) [production]
08:07 <jmm@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1122', diff saved to https://phabricator.wikimedia.org/P9088 and previous config saved to /var/cache/conftool/dbconfig/20190911-080450-marostegui.json [production]
07:52 <moritzm> reimaging restbase-dev1005 to Stretch T224554 [production]
07:51 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1122', diff saved to https://phabricator.wikimedia.org/P9087 and previous config saved to /var/cache/conftool/dbconfig/20190911-075139-marostegui.json [production]
07:33 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1122', diff saved to https://phabricator.wikimedia.org/P9086 and previous config saved to /var/cache/conftool/dbconfig/20190911-073335-marostegui.json [production]
07:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1122', diff saved to https://phabricator.wikimedia.org/P9085 and previous config saved to /var/cache/conftool/dbconfig/20190911-072344-marostegui.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1122', diff saved to https://phabricator.wikimedia.org/P9084 and previous config saved to /var/cache/conftool/dbconfig/20190911-071450-marostegui.json [production]
07:07 <marostegui> Stop MySQL on db1122 to reboot for a kernel upgrade T230785 [production]
07:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1122 to reboot for kernel upgrade T230785', diff saved to https://phabricator.wikimedia.org/P9083 and previous config saved to /var/cache/conftool/dbconfig/20190911-070635-marostegui.json [production]
07:00 <hashar> Restarting Gerrit - T224448 [production]
06:58 <hashar> Restarting Gerrit [production]
06:45 <marostegui> Drop unused database puppet on m1 - T231539 [production]
06:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Re-organize s1 codfw weights and roles - T230106', diff saved to https://phabricator.wikimedia.org/P9082 and previous config saved to /var/cache/conftool/dbconfig/20190911-061924-marostegui.json [production]
06:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Re-organize s1 codfw weights and roles - T230106', diff saved to https://phabricator.wikimedia.org/P9081 and previous config saved to /var/cache/conftool/dbconfig/20190911-061659-marostegui.json [production]
05:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2048, will be decommissioned T230106', diff saved to https://phabricator.wikimedia.org/P9080 and previous config saved to /var/cache/conftool/dbconfig/20190911-054855-marostegui.json [production]
05:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote db2112 to s1 codfw master T230106', diff saved to https://phabricator.wikimedia.org/P9079 and previous config saved to /var/cache/conftool/dbconfig/20190911-054753-marostegui.json [production]
05:29 <marostegui> Switchover s1 codfw master db2048 -> db2112 T230106 [production]
03:31 <eileen> civicrm revision changed from b343642c76 to 53aeba6318, config revision is 3e22a80bc8 [production]
2019-09-10 §
20:46 <ejegg> updated payments-wiki from 15baf7f58b to 5432f9c3a4 [production]
20:24 <XioNoX> add MSS clamp on install1002 - T2324563 [production]
20:20 <XioNoX> add MSS clamp on archiva1001 - T232456 [production]
18:42 <herron> rolling out "Aggregate IPsec Tunnel Status” icinga check, please disregard for the time being if it alerts [production]
18:15 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T229863 Remove EventBusRCFeedEngine eventServiceName (duration: 01m 05s) [production]
18:15 <XioNoX> rollback test add static route on bast3002 to force advmss [production]