501-550 of 10000 results (74ms)
2019-05-30 §
18:11 <Krinkle> mw1348 (recent api/php72 100% experiment) shows signs of corruption [production]
18:11 <Krinkle> mw1321 php7.2 shows signs of corruption for over 2 hours – https://phabricator.wikimedia.org/T224491#5224464 [production]
18:03 <krinkle@deploy1001> Synchronized wmf-config/arclamp.php: (no justification provided) (duration: 00m 53s) [production]
16:24 <bblack> re-pool cp3047 into service as ats-be - T222937 [production]
16:04 <mutante> phab1001 - removing 2620:0:861:103:10:64:32:186/128 from eth0 [production]
16:03 <mutante> phab1001 - removing 10.64.32.186/32 from eth0 [production]
16:01 <mutante> phab1001 - removing git-ssh.wm.org IP from interface - phab1003 - activating IPv6 listen address for git-ssh [production]
15:36 <jynus> stop es1019 for maintenance T213422 [production]
15:26 <cmjohnson1> shutting down db1099 to swap DIMM T221502 [production]
15:20 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1089 with full weight; depool es1019 (duration: 00m 52s) [production]
15:19 <herron> performing rolling reboots of eqiad kafka main cluster hosts for security updates [production]
15:06 <onimisionipe> pooled maps2004 - osm import is complete - T224395 [production]
14:44 <andrewbogott> reimaging cloudvirtan1001 for T224566 [production]
14:43 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:43 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:42 <andrewbogott> reimaging cloudvirtan1001 [production]
14:29 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:29 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:25 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:25 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:22 <bblack> rebooting cp3047 (post-reimage/puppetization for T222937) [production]
14:21 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:21 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:05 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:05 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:00 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:00 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:57 <jijiki> enable puppet on mw* in eqiad [production]
13:44 <volans> rm /root/.ssh/known_hosts on cumin[12]001 [production]
13:40 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:40 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:36 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.7 [production]
13:28 <jijiki> Enabling puppet on mw*.codfw.net [production]
13:22 <zfilipin@deploy1001> Synchronized php-1.34.0-wmf.7/resources/src/jquery/jquery.suggestions.js: SWAT: [[gerrit:513237|jquery.suggestions: Do not show suggestions on prefilled values ([T224524])]] (duration: 00m 58s) [production]
13:17 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1015.eqiad.wmnet [production]
13:17 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1014.eqiad.wmnet [production]
13:17 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1013.eqiad.wmnet [production]
13:17 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1012.eqiad.wmnet [production]
13:17 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1011.eqiad.wmnet [production]
13:16 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1010.eqiad.wmnet [production]
13:16 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1009.eqiad.wmnet [production]
13:16 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1008.eqiad.wmnet [production]
13:16 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=restbase1007.eqiad.wmnet [production]
13:08 <bblack> cp3047 puppet-disable + depool for reimage to ATS - T222937 [production]
13:03 <marostegui> Stop MySQL on db1099 for onsite maintenance - T221502 [production]
13:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1099 T221502 (duration: 00m 56s) [production]
13:00 <reedy@deploy1001> Synchronized php-1.34.0-wmf.7/tests/phpunit/includes/: T222628 (duration: 01m 06s) [production]
12:58 <reedy@deploy1001> Synchronized php-1.34.0-wmf.7/includes/Linker.php: T222628 (duration: 01m 04s) [production]
12:49 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:49 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]