401-450 of 10000 results (54ms)
2019-09-16 §
05:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2054 from config T232969 (duration: 01m 05s) [production]
2019-09-15 §
16:51 <Krinkle> Fixed a dozen abuse filtes, listed at https://phabricator.wikimedia.org/T156096#5494060. The trailing pipe character was removed from filters that had it which is no longer supported in a future version of AbuseFilter. [production]
14:35 <_joe_> test: setting opcache.interned_strings_buffer to 0 on mw1348 for T232613 [production]
2019-09-14 §
23:42 <onimisionipe> force shard allocation (dewiki_content_1566659363[4]) on eqiad cluster [production]
04:39 <effie> Depool and reload mw1286 [production]
01:14 <ejegg> updated fundraising python tools from 1e405864d7 to e1b81688c6 [production]
00:29 <ejegg> updated payments-wiki from 1f556670cf to fc82318180 [production]
2019-09-13 §
23:44 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:42 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:06 <gehel> re-enable puppet on maps - T232817 [production]
20:23 <chaomodus> restarting netbox1001.wikimedia.org [production]
20:00 <twentyafterfour> hotfixing T232600 due to severity of the bug and relative safety of the fix (if this breaks, yell at James_F who twisted my arm and made me do it) [production]
19:54 <urandom> bootstrapping Cassandra, restbase2009-c -- T224553 [production]
17:24 <urandom> bootstrapping Cassandra, restbase2009-b -- T224553 [production]
16:10 <XioNoX> fix bgp group netflow on cr2-codfw [production]
15:47 <urandom> bootstrapping Cassandra, restbase2009-a -- T224553 [production]
15:43 <effie> reverting live hacks on mw1348 [production]
15:34 <hashar@deploy1001> Synchronized wmf-config/CommonSettings.php: Disable adhoc core dump logging - T232613 (duration: 01m 04s) [production]
15:14 <akosiaris> upload apertium-dan_0.6.0-1+wmf3 apertium-nno_1.0.0-1+wmf1 apertium-nob_1.0.0-2+wmf1 apertium-swe_0.8.0-1+wmf1 to apt.wikimedia.org/jessie-wikimedia T218184 [production]
15:11 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:11 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:08 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:08 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:02 <hashar@deploy1001> Synchronized php-1.34.0-wmf.22/includes/libs/rdbms/lbfactory/LBFactoryMulti.php: Add more log and context for T232613 logging - T232613 (duration: 01m 04s) [production]
15:02 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:02 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:51 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:51 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:37 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:36 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:30 <akosiaris@> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
14:30 <moritzm> installing cups security update on buster (only client-side libs installed) [production]
14:22 <moritzm> installing bzip2 update from Buster 10.1 point release [production]
14:18 <moritzm> installing reportbug update from Buster 10.1 point release [production]
14:14 <akosiaris@> helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
14:05 <akosiaris@> helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
13:57 <oblivian@deploy1001> Synchronized wmf-config/logging.php: unbreak mediawiki logging on scandium (duration: 01m 04s) [production]
13:28 <akosiaris@> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
13:27 <akosiaris@> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:21 <akosiaris@> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
13:20 <akosiaris@> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
13:19 <akosiaris@> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
12:56 <_joe_> banning more urls on maps1003 [production]
12:37 <_joe_> temp ban of class of urls on maps1003 nginx [production]
12:14 <jbond42> add timing information to maps1003 access logs [production]
11:39 <jbond42> enable access logs on maps1003 [production]
11:38 <_joe_> manually raising the worker heap limit to 600 MB on kartotherian on maps1003 [production]
11:11 <elukey> reboot an-conf100* (Analytics Zookeeper nodes - not yet in production) for kernel upgrades [production]
11:10 <elukey> reboot an-tool1007 (runs turnilo) for kernel upgrades [production]
11:08 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]