5201-5250 of 10000 results (34ms)
2020-07-09 §
11:50 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
11:50 <moritzm> rebooting debmonitor1001 for kernel update [production]
11:42 <urbanecm@deploy1001> Synchronized php-1.35.0-wmf.40/extensions/Translate/tag/SpecialPageTranslation.php: 6541d3ff51f52fe8a1bdbfa86022f8d97d6c7680: DeprecatablePropertyArray: Use MW_VERSION instead of array_key_exists (T257531) (duration: 01m 05s) [production]
11:28 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 3a7c1c33e58637437f819edf039008a00dc5be27: Rename namespace on kn.wikipedia.org (T255337) (duration: 01m 04s) [production]
11:24 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 0a3c1f94a702b527842ed4f34d8bf41b26235e64: Add *.oireachtas.ie to the wgCopyUploadsDomains whitelist for commonswiki (T256543) (duration: 01m 04s) [production]
11:19 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
11:17 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
11:10 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:10 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:09 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: e6f442c6900524482806aeb1b5162e65bf7c97ac: Enable Quicksurveys for Desktop Improvements Project (T246977) (duration: 01m 06s) [production]
11:01 <vgutierrez> restart ats-tls on cp1085 [production]
10:55 <_joe_> restarting php7.2-fpm on mw1282, workers failing with sigill [production]
10:54 <_joe_> depool mw1282 [production]
10:54 <mvolz@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:34 <mvolz@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:23 <_joe_> rolling restart the remaining restbases in eqiad, and all of codfw [production]
10:22 <mvolz@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'citoid' for release 'staging' . [production]
10:09 <_joe_> restarting restbase on rb1020-22 [production]
09:53 <_joe_> restarting restbase on restbase1024,1023 [production]
09:36 <_joe_> restarting restbase on rb1026,1027 to switch to proton on k8s [production]
09:34 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:31 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:28 <_joe_> restarting restbase on restbase1025 to pick up the switch to k8s of proton [production]
09:27 <godog> bounce thanos-compact on thanos-fe2001 [production]
09:07 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.change-distro (exit_code=0) [production]
08:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1079', diff saved to https://phabricator.wikimedia.org/P11828 and previous config saved to /var/cache/conftool/dbconfig/20200709-085228-marostegui.json [production]
08:44 <marostegui> Stop haproxy on dbproxy1017 before upgrading to buster - T255408 [production]
08:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1136', diff saved to https://phabricator.wikimedia.org/P11827 and previous config saved to /var/cache/conftool/dbconfig/20200709-082355-marostegui.json [production]
08:23 <moritzm> imported osm2pgsql 0.96.0+ds-1~bpo9+1 to "main" component T256877 [production]
08:22 <elukey@cumin1001> START - Cookbook sre.hadoop.change-distro [production]
08:20 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.stop-cluster (exit_code=0) [production]
08:13 <elukey@cumin1001> START - Cookbook sre.hadoop.stop-cluster [production]
08:11 <XioNoX> disable igmp snooping on msw1-codfw [production]
07:59 <marostegui> Stop db1117:3322 to clone db1084, this will trigger haproxy alerts - T257540 [production]
07:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1136', diff saved to https://phabricator.wikimedia.org/P11825 and previous config saved to /var/cache/conftool/dbconfig/20200709-075749-marostegui.json [production]
07:19 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:15 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:52 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:49 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
05:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1101:3317', diff saved to https://phabricator.wikimedia.org/P11824 and previous config saved to /var/cache/conftool/dbconfig/20200709-053905-marostegui.json [production]
05:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1084 from dbctl', diff saved to https://phabricator.wikimedia.org/P11823 and previous config saved to /var/cache/conftool/dbconfig/20200709-053206-marostegui.json [production]
05:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1084', diff saved to https://phabricator.wikimedia.org/P11822 and previous config saved to /var/cache/conftool/dbconfig/20200709-051826-marostegui.json [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1101:3317', diff saved to https://phabricator.wikimedia.org/P11821 and previous config saved to /var/cache/conftool/dbconfig/20200709-051355-marostegui.json [production]
05:11 <marostegui> Remove revision triggers from db2093:3315 T238966 [production]
05:10 <marostegui> Deploy schema change on s5 codfw, lag will be generated - T238966 [production]
01:43 <tzatziki> reset email for GseSro [production]
00:58 <cdanis> ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 sudo cumin A:cp 'enable-puppet "cdanis deploying I6c1b646e T256395"' [production]
00:49 <cdanis> ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 sudo cumin A:cp 'disable-puppet "cdanis deploying I6c1b646e T256395"' [production]
2020-07-08 §
21:56 <mutante> deleting files from releases2001 that are not existing on releases1001 to make them mirrors. rsync with --delete and the command from quickdatacopy class (T247652) [production]
21:55 <mutante> rsyncing releases files from releases1001 to releases2002 and releases1002. deleting files from releases2002 not existing on releases1002 to make them mirrors ( T247652_ [production]