4151-4200 of 10000 results (37ms)
2020-09-24 ยง
14:09 <cmjohnson1> removing the cable connected to FPC1:1/0 (DAC 3m) FPC8:1/0 (DAC 3m) [production]
13:58 <moritzm> upgrading mariadb on cloudcontrol-2001/2003/2004 [production]
13:52 <XioNoX> depool eqiad for row D recabling - T256112 [production]
13:32 <ottomata> Increased retention time for *.mediawiki.job.processMediaModeration topics in kafka main-eqiad and main-codfw to 31 days (as per request from Pchelolo ) [production]
13:22 <elukey> moved the hadoop cluster to puppet TLS certificates - T253957 [production]
13:17 <XioNoX> add damping to anycast BGP - T262372 [production]
12:58 <jayme> switched mathoid service-proxy listener to use TLS - T255875 [production]
12:50 <moritzm> upgrading bird on centtrallog1001 [production]
12:43 <gehel> restarting wdqs-categories on wdqs1009 [production]
12:43 <moritzm> installing netty-3.9 security updates [production]
12:42 <jgiannelos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
12:30 <ema> upload@ulsfo: rolling varnish upgrade to 6.0.6-1wm1 T263557 [production]
12:29 <godog> swift codfw-prod: rebalance only, no weight change [production]
12:27 <kormat> powering off db2125 for maintenance T260670 [production]
12:25 <moritzm> installing xorg-server security updates [production]
12:09 <ema> text@ulsfo: rolling varnish upgrade to 6.0.6-1wm1 T263557 [production]
12:02 <ema> cp4022: upgrade varnish to 6.0.6-1wm1 T263557 [production]
11:40 <Urbanecm> EU B&C window done [production]
11:34 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.10/extensions/Translate/tag/TPSection.php: fa4900e1e6022e645be12505de30b696a9769e77: Fix validation of translation unit section names (T263546) (duration: 01m 07s) [production]
11:25 <jbond42> re-enable puppet fleet wide [production]
11:22 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: fdab74c443bc3328856e8441f4d2df8bc57c6f54: Enable ContentTranslation in Bashkir, Urdu and Welsh WPs as a default tool (T258504; T260022; T260024) (duration: 01m 05s) [production]
11:21 <jbond42> disable puppet fleet wide to reduce log level on puppetdb [production]
11:14 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 90c72912f26d91df6d28b1efd64e366aaabc5357: Move DiscussionTools out of beta on arwiki, cswiki, huwiki (T249394); d8553f35b4dd581f67bd568d773ff65f316fbfd3: Simplify DiscussionTools config (duration: 01m 11s) [production]
11:06 <moritzm> installing imagemagick security updates on stretch [production]
11:02 <jbond42> re-enable puppet fleet wide [production]
10:51 <jbond42> disable puppet fleet wide to deploy a puppetmaster change [production]
10:49 <moritzm> installing libproxy security updates [production]
10:23 <volans> uploaded python3-wmflib_0.0.2 to apt.wikimedia.org buster-wikimedia [production]
10:20 <kormat@cumin1001> dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12789 and previous config saved to /var/cache/conftool/dbconfig/20200924-102025-kormat.json [production]
10:05 <kormat@cumin1001> dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12788 and previous config saved to /var/cache/conftool/dbconfig/20200924-100521-kormat.json [production]
10:02 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
10:01 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
09:50 <kormat@cumin1001> dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12787 and previous config saved to /var/cache/conftool/dbconfig/20200924-095018-kormat.json [production]
09:50 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
09:50 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
09:48 <jayme> restart pybal on lvs1015.eqiad.wmnet,lvs2009.codfw.wmnet - T255875 [production]
09:46 <jayme> restart pybal on lvs1016.eqiad.wmnet,lvs2010.codfw.wmnet - T255875 [production]
09:43 <jayme> running puppet on lvs servers - T255875 [production]
09:35 <kormat@cumin1001> dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 25%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12786 and previous config saved to /var/cache/conftool/dbconfig/20200924-093514-kormat.json [production]
09:25 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
09:25 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
09:20 <ema> cp4021: repool with varnish 6.0.6-1wm1 T263557 [production]
09:19 <ema> cp4021: redepool with varnish to 6.0.6-1wm1 T263557 [production]
09:14 <kormat@cumin1001> dbctl commit (dc=all): 'db2138:3312 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12785 and previous config saved to /var/cache/conftool/dbconfig/20200924-091445-kormat.json [production]
09:14 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:14 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:14 <ema> cp4021: depool and upgrade varnish to 6.0.6-1wm1 T263557 [production]
09:05 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
09:04 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
08:59 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]