501-550 of 10000 results (22ms)
2020-05-05 §
11:26 <moritzm> rolling restart of apache/FPM on mw1261-mw1265 [production]
11:22 <kart_> EU SWAT done. [production]
11:09 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|592479|Adjust ContentTranslation MT threshold for Chinese WP to 70% (T246383)]] (duration: 01m 01s) [production]
11:01 <moritzm> installing remaining openldap security updates (client-side libs, tools) [production]
11:00 <kormat@cumin1001> dbctl commit (dc=all): 'Depool es1024 for reimaging, add es1023 (master) for reading in the meantime T250666', diff saved to https://phabricator.wikimedia.org/P11141 and previous config saved to /var/cache/conftool/dbconfig/20200505-110031-kormat.json [production]
10:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1126 T248086', diff saved to https://phabricator.wikimedia.org/P11140 and previous config saved to /var/cache/conftool/dbconfig/20200505-104540-marostegui.json [production]
10:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1126 T248086', diff saved to https://phabricator.wikimedia.org/P11139 and previous config saved to /var/cache/conftool/dbconfig/20200505-104441-marostegui.json [production]
10:33 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
10:23 <arturo> copy prometheus-rabbitmq-exporter v0.4 from stretch-wikimedia to buster-wikimedia in apt1001 (T251660) [production]
10:18 <arturo> copy prometheus-pdns-exporter v0.5.1 from stretch-wikimedia to buster-wikimedia in apt1001 (T251575) [production]
10:16 <mutante> temp disabling puppet on all ganeti hosts to carefully deploy change related to rapi cert location [production]
09:37 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
09:36 <moritzm> removing boron.eqiad.wmnet [production]
09:36 <jmm@cumin2001> START - Cookbook sre.hosts.decommission [production]
09:03 <gehel> restarting wdqs updater on all servers [production]
08:53 <moritzm> installing Java security updates on releases* [production]
08:44 <kormat> reimaging es1024 to buster T250666 [production]
08:27 <ema> cp2028 and cp2030 (both upload): varnish-fe restart to clear cache and evaluate 'exp' admission policy T144187 T249809 [production]
08:26 <moritzm> upgrading slapd on serpens/seaborgium [production]
08:19 <ema> cp2027 and cp2029 (both text): varnish-fe restart to clear cache and evaluate 'exp' admission policy T144187 T249809 [production]
08:08 <moritzm> installing Java security updates on notebook/stat hosts [production]
07:54 <gehel@deploy1001> Finished deploy [wdqs/wdqs@d37a059]: rollback wdqs to v 0.3.22 (duration: 04m 18s) [production]
07:50 <gehel@deploy1001> Started deploy [wdqs/wdqs@d37a059]: rollback wdqs to v 0.3.22 [production]
07:36 <zpapierski@deploy1001> Started deploy [wdqs/wdqs@d37a059]: fix for the duplicated jars [production]
06:59 <addshore> depool wdqs1006 heavy lag [production]
05:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Set s5 and s6 as read-only=off for maintenance T251154', diff saved to https://phabricator.wikimedia.org/P11133 and previous config saved to /var/cache/conftool/dbconfig/20200505-052334-marostegui.json [production]
05:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Set s5 and s6 as read-only for maintenance T251154', diff saved to https://phabricator.wikimedia.org/P11132 and previous config saved to /var/cache/conftool/dbconfig/20200505-052058-marostegui.json [production]
05:19 <marostegui> Start s5 and s6 maintenance - T251154 [production]
04:39 <marostegui> Restart mysql on tendril host: db1115 - T231769 [production]
2020-05-04 §
23:38 <mstyles@deploy1001> Finished deploy [wdqs/wdqs@6518a8d]: v.0.3.26 (duration: 14m 39s) [production]
23:37 <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Use namespaced EventBus classes (duration: 00m 57s) [production]
23:35 <reedy@deploy1001> Synchronized wmf-config/logging.php: Use namespaced EventBus classes (duration: 00m 56s) [production]
23:33 <reedy@deploy1001> Synchronized rpc/RunSingleJob.php: Use namespaced EventBus classes (duration: 00m 58s) [production]
23:29 <reedy@deploy1001> Synchronized wmf-config/logging.php: Replace AuthManagerStatsdHandler with WikimediaEventsAuthManagerStatsdHandler::class (duration: 00m 57s) [production]
23:23 <mstyles@deploy1001> Started deploy [wdqs/wdqs@6518a8d]: v.0.3.26 [production]
22:42 <sbassett@deploy1001> Synchronized private/PrivateSettings.php: T251835: Restore dc752af1e94684faacbe9662789815c6edbbdf46 (duration: 00m 57s) [production]
22:16 <eileen> process-control config revision is 2eb75f8dff [production]
22:06 <sbassett@deploy1001> Synchronized private/PrivateSettings.php: Partial mitigation for T250887 (duration: 00m 57s) [production]
21:45 <sbassett@deploy1001> Synchronized private/PrivateSettings.php: Revert partial mitigation for T250887 (duration: 00m 57s) [production]
21:41 <sbassett@deploy1001> Synchronized private/PrivateSettings.php: Deploy partial mitigation for T250887 (duration: 00m 57s) [production]
18:20 <dpifke@deploy1001> Finished deploy [performance/navtiming@239d359]: Deploy navtiming with new/updated Prometheus metrics - T249822, T238086 (duration: 00m 05s) [production]
18:19 <dpifke@deploy1001> Started deploy [performance/navtiming@239d359]: Deploy navtiming with new/updated Prometheus metrics - T249822, T238086 [production]
18:16 <Urbanecm> Morning SWAT done [production]
18:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: c04fbdd: Adding upload_by_url user right to all registered users on Commons (T251474) (duration: 00m 57s) [production]
18:11 <urbanecm@deploy1001> Synchronized php-1.35.0-wmf.30/extensions/DiscussionTools/includes/DiscussionToolsHooks.php: SWAT: b85fc16: Enable on all ExtraSignaturesNamespaces (T249036) (duration: 01m 00s) [production]
18:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 18c1efb: Load DiscussionTools on en.wiki (T249376) (duration: 00m 58s) [production]
17:57 <XioNoX> configure singtel interface on cr1-eqsin [production]
17:36 <volans> upgraded spicerack on cumin[12]001 to 0.0.33-1 [production]
17:02 <joal@deploy1001> Finished deploy [analytics/refinery@2252f9a] (thin): Analytics hotfix deploy 2 THIN (sqoop) [2252f9a] (duration: 00m 09s) [production]
17:02 <joal@deploy1001> Started deploy [analytics/refinery@2252f9a] (thin): Analytics hotfix deploy 2 THIN (sqoop) [2252f9a] [production]