651-700 of 10000 results (59ms)
2019-07-15 §
08:22 <godog> set oemhp_powerreg=os on ms-be10[16-39] - T225713 [production]
08:01 <vgutierrez> upgrading acme-chief to version 0.19 in acme-chief production instances - T225945 [production]
2019-07-14 §
13:18 <godog> silence mr1-eqsin.oob IPv6 until tomorrow 8 UTC - T227967 [production]
12:01 <Urbanecm> Running mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Sporti /home/urbanecm/T227968 for server side upload [production]
2019-07-13 §
01:51 <MaxSem> DIsabled 2FA for my staff account [production]
2019-07-12 §
23:35 <mutante> netmon1003 - shutdown -h now after it's gone from Icinga now [production]
23:31 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:31 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
23:28 <mutante> netmon1003 - stopping apache2 service (decom of servermon.wikimedia.org) [production]
19:41 <James_F> Disabled 2FA for MSchottlender-WMF for device reset. [production]
19:17 <shdubsh> add prometheus-varnishkafka-exporter 0.1 to apt repo T196066 [production]
19:15 <urandom> bootstrapping restbase1017-c -- T222960 [production]
19:08 <jeh> rebooting cloudvirt1018.eqiad.wmnet T216040 [production]
18:53 <mutante> cp1072 - enabling notifications for service checks in icinga, they were disabled but all green and no SAL/ticket. looked like forgotten from the past [production]
18:49 <gehel> setting CPU governor to performance for wdqs1010 - T225713 [production]
18:16 <Krinkle> Remove bogus Graphite data at frontend.navtiming2.requet (typo from Nov 2018), graphite1004/2003 [production]
18:02 <urandom> bootstrapping restbase1017-b -- T222960 [production]
16:32 <urandom> bootstrapping restbase1017-a -- T222960 [production]
16:25 <jijiki> Rolling restart swift proxy on ms-fe* [production]
15:25 <jeh> rebooting cloudvirt1018.eqiad.wmnet T216040 [production]
14:05 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
12:45 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
12:39 <fsero> recreating ci staging namespaces T227775 [production]
12:39 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . [production]
12:38 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . [production]
12:36 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . [production]
12:33 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
12:33 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
12:22 <fsero> recreating eventgate-* and blubberoid staging namespaces T227775 [production]
12:22 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . [production]
12:22 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:15 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'sessionstore' for release 'staging' . [production]
12:11 <fsero> recreating sessionstore,cxserver and mathoid staging namespaces T227775 [production]
12:10 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'citoid' for release 'staging' . [production]
12:06 <fsero> recreating citoid staging namespace T227775 [production]
12:05 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'staging' . [production]
12:01 <fsero> recreating termbox staging namespace T227775 [production]
11:09 <jynus@deploy1001> Synchronized wmf-config/db-codfw.php: Switchover db2045 x1 codfw master to db2069 (duration: 00m 51s) [production]
10:24 <jynus> switchover x1 codfw master from db2045 to db2069 T227862 [production]
10:23 <jynus> switchover x1 codfw master from db2045 to db2069 [production]
09:43 <moritzm> shut down ldap-codfw-replica01/ldap-codfw-replica02 (pending reimage) [production]
08:18 <jijiki> enable puppet on mw1222 [production]
06:35 <vgutierrez> upgrading acme-chief to version 0.19 in acme-chief test instances - T225945 [production]
06:28 <vgutierrez> uploaded acme-chief 0.19 to apt.wikimedia.org (buster) - T225945 [production]
05:45 <elukey> sudo -i /usr/local/sbin/restart-php7.2-fpm on mwdebug* to clear opcache [production]
01:01 <Krinkle> mw1342 generated some ~ 11,500 additional PHP errors over a 4 hour period (18:00-22:30 UTC), ref T224491 [production]
00:59 <Krinkle> mw1342 is generating strange PHP erros (php7 only), ref T224491 [production]