651-700 of 10000 results (67ms)
2019-07-15 §
08:50 <fsero> creating docker_registry_codfw on eqiad T227570 [production]
08:49 <gehel> correction: set oemhp_powerreg=os + reboot for elastic1052 (NOT elastic1054) - T225713 [production]
08:48 <fsero> T227570 changing container_synchronization on docker_registry_codfw to //docker_registry/eqiad/AUTH_docker/docker_registry_codfw [production]
08:48 <gehel> set oemhp_powerreg=os + reboot for elastic1054 - T225713 [production]
08:22 <godog> set oemhp_powerreg=os on ms-be10[16-39] - T225713 [production]
08:01 <vgutierrez> upgrading acme-chief to version 0.19 in acme-chief production instances - T225945 [production]
06:50 <elukey> run msck repair table mediawiki_wikitext_history in beeline [analytics]
2019-07-14 §
13:18 <godog> silence mr1-eqsin.oob IPv6 until tomorrow 8 UTC - T227967 [production]
12:01 <Urbanecm> Running mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Sporti /home/urbanecm/T227968 for server side upload [production]
00:55 <Krinkle> krinkle@doc1001: Remove https://doc.wikimedia.org/cover-extensions/$DOC_BASENAME/ - a failed build for GrowthExperiments [releng]
2019-07-13 §
14:31 <wm-bot> <maurelio> Updated stewardbots to 69faa7e [index: Display git hash and last modified]. [tools.stewardbots]
11:42 <wm-bot> <maurelio> Restarted webservice due to tools not loading. [tools.meta]
01:51 <MaxSem> DIsabled 2FA for my staff account [production]
2019-07-12 §
23:35 <mutante> netmon1003 - shutdown -h now after it's gone from Icinga now [production]
23:31 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:31 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
23:28 <mutante> netmon1003 - stopping apache2 service (decom of servermon.wikimedia.org) [production]
23:07 <James_F> Zuul: Instead of hacking around for WebAuthn, define bespoke quibble php72+ template [releng]
22:10 <James_F> Zuul: Make seccheck voting on WebAuthn T226737 [releng]
20:30 <andrewbogott> "apt-get install --reinstall libpython3.4-stdlib" on puppetmaster-01; there was a syntax error in /usr/lib/python3.4/http/client.py which was breaking lots of things. Don't know how it got there. [wikifactmine]
19:41 <James_F> Disabled 2FA for MSchottlender-WMF for device reset. [production]
19:28 <James_F> Zuul: Add MobileFrontend and MinervaNeue to Wikibase's dependencies [releng]
19:17 <shdubsh> add prometheus-varnishkafka-exporter 0.1 to apt repo T196066 [production]
19:15 <urandom> bootstrapping restbase1017-c -- T222960 [production]
19:08 <jeh> rebooting cloudvirt1018.eqiad.wmnet T216040 [production]
18:53 <mutante> cp1072 - enabling notifications for service checks in icinga, they were disabled but all green and no SAL/ticket. looked like forgotten from the past [production]
18:49 <gehel> setting CPU governor to performance for wdqs1010 - T225713 [production]
18:16 <Krinkle> Remove bogus Graphite data at frontend.navtiming2.requet (typo from Nov 2018), graphite1004/2003 [production]
18:02 <urandom> bootstrapping restbase1017-b -- T222960 [production]
16:56 <James_F> Zuul: Add Flow as phan dependency to GrowthExperiments [releng]
16:32 <urandom> bootstrapping restbase1017-a -- T222960 [production]
16:25 <jijiki> Rolling restart swift proxy on ms-fe* [production]
15:25 <jeh> rebooting cloudvirt1018.eqiad.wmnet T216040 [production]
14:05 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
12:45 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
12:39 <fsero> recreating ci staging namespaces T227775 [production]
12:39 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . [production]
12:38 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . [production]
12:36 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . [production]
12:33 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
12:33 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . [production]
12:22 <fsero> recreating eventgate-* and blubberoid staging namespaces T227775 [production]
12:22 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . [production]
12:22 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:18 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
12:15 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'sessionstore' for release 'staging' . [production]
12:11 <fsero> recreating sessionstore,cxserver and mathoid staging namespaces T227775 [production]
12:10 <fsero@> helmfile [STAGING] Ran 'apply' command on namespace 'citoid' for release 'staging' . [production]