751-800 of 10000 results (53ms)
2019-04-08 §
07:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool all slaves in x1 T143763 (duration: 00m 58s) [production]
06:34 <_joe_> restarted netbox, SIGSEGV on HUP-induced reload [production]
05:20 <marostegui> Deploy schema change on x1 master with replication, there will be lag on x1 slaves T143763 [production]
05:18 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool all slaves in x1 T219777 T143763 (duration: 01m 30s) [production]
2019-04-07 §
11:34 <volans|off> restarted icinga on icinga2001 [production]
06:34 <oblivian@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=zotero,name=codfw [production]
06:23 <_joe_> deleting zotero pods with high memory watermark in codfw [production]
06:03 <oblivian@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=zotero,name=codfw [production]
2019-04-06 §
10:09 <gilles> Purging ruwiki namespaces > 0 [production]
2019-04-05 §
23:10 <thcipriani> revert some recent problematic gerrit acl changes [production]
22:46 <chaomodus> restarted pdfrender on scb1002 T174916 [production]
21:45 <hashar> thcipriani restarted Gerrit. CI works again # T220243 [production]
21:37 <thcipriani> restarting gerrit [production]
21:29 <hashar> CI / Zuul is no more processing events / T220243 [production]
17:29 <thcipriani> gerrit back on 2.15.11 [production]
17:27 <thcipriani> restart gerrit [production]
17:26 <thcipriani@deploy1001> Finished deploy [gerrit/gerrit@a4e66d4]: Gerrit to back to 2.15.11 on cobalt (restart incoming) (duration: 00m 11s) [production]
17:26 <thcipriani@deploy1001> Started deploy [gerrit/gerrit@a4e66d4]: Gerrit to back to 2.15.11 on cobalt (restart incoming) [production]
17:25 <thcipriani@deploy1001> Finished deploy [gerrit/gerrit@a4e66d4]: Gerrit to back to 2.15.11 (on gerrit2001 only) (duration: 00m 10s) [production]
17:25 <thcipriani@deploy1001> Started deploy [gerrit/gerrit@a4e66d4]: Gerrit to back to 2.15.11 (on gerrit2001 only) [production]
17:19 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.23/includes/diff/TextSlotDiffRenderer.php: Ia326c67de28a4e / T220217 (duration: 01m 02s) [production]
17:12 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.24/includes/diff/TextSlotDiffRenderer.php: Ia326c67de28a4e / T220217 (duration: 01m 00s) [production]
16:02 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.24/includes/jobqueue/jobs/RefreshLinksJob.php: Ib1ac31365f9c / T220037 (duration: 00m 59s) [production]
15:58 <ejegg> re-enabled recurring donations queue consumer [production]
15:57 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/NavigationTiming/: I6b23be850d35c7d19 / T220156 (duration: 01m 00s) [production]
15:51 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/GlobalBlocking/includes/specials/: I5843cd181ca7d (duration: 01m 02s) [production]
15:08 <ejegg> upgraded fundraising CiviCRM from 3c55850631 to 83478013a8 [production]
15:01 <ejegg> disabled recurring donation queue consumer [production]
14:55 <papaul> powering down restbase2019 and 2020 for relocation [production]
13:53 <gehel@cumin2001> END (FAIL) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=99) [production]
13:45 <akosiaris> repool eqiad for all kubernetes services T217426 [production]
13:45 <akosiaris> ρepool eqiad for all kubernetes services T217426 [production]
13:45 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=citoid [production]
13:45 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=cxserver [production]
13:45 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=mathoid [production]
13:44 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=blubberoid [production]
13:44 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=zotero [production]
13:41 <arturo> T220203 reimage labtestnet2002 as spare in stretch [production]
13:36 <arturo> T220101 disable active icinga checks for cloudcontrol2002-dev [production]
13:35 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:35 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
13:35 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:35 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
12:50 <gehel@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=99) [production]
12:49 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
12:48 <jijiki> Restarting pybal on lvs1016 and lvs2003 for 496382 [production]
12:43 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
12:43 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
12:43 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
12:43 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]