2019-04-26
§
|
20:15 |
<foks> |
changing email and password for "Lemon martini@global" |
[production] |
19:38 |
<foks> |
changing password for JDiPierro@global |
[production] |
19:21 |
<bblack> |
varnish-backend-restart on cp4026, evidence of artificial 503s from mbox lag behavior, probably related to the semi-abuse client doing odd 404 traffic to ulsfo that's triggering bugs in swift's rewrite.py .... |
[production] |
19:04 |
<foks> |
changing password for Subinsebastien |
[production] |
17:50 |
<mutante> |
analytics1052 - reported broken systemd state in Icinga - service mcelog was in state failed - systemctl start mcelog - (T212219 ?) |
[production] |
16:18 |
<jynus> |
stop s6 mariadb instance on dbstore2001 T220572 |
[production] |
15:34 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: thumbor1001 ms-fe1005 ms-be1013 scb1001 restbase1007 |
[production] |
15:05 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: ores1001.yaml wtp1025.yaml rdb1006.yaml |
[production] |
14:18 |
<marostegui> |
Set pc1004-1006 and pc2004-2006 as unracked on netbox - T209858 T210969 |
[production] |
13:17 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: mw1311.yaml, mx2001 & dubnium |
[production] |
12:52 |
<ema> |
cp4025: restart varnish-be due to mbox lag |
[production] |
12:50 |
<jijiki> |
Restarting hhvm on mw1288 |
[production] |
12:48 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on mc1019, maps1001 and logstash1007 |
[production] |
12:45 |
<ema@puppetmaster1001> |
conftool action : set/pooled=yes; selector: cluster=cache_upload,name=cp4021.ulsfo.wmnet,dc=ulsfo |
[production] |
12:44 |
<ema> |
pool cp4021 w/ ATS backend T219967 |
[production] |
12:20 |
<ema> |
repool cp3030 after directors.frontend.vcl testing T219967 |
[production] |
12:09 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: elastic1017, ganeti2001, analytics1042 |
[production] |
11:26 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on lvs4007, dns2001 and multatuli |
[production] |
11:16 |
<jbond42> |
upgrade puppet 4=> 5 and facter 2 => 3 on bast4002, aqs1004 and conf2001 |
[production] |
10:28 |
<moritzm> |
restarting Parsoid on wtp1025 for glibc update |
[production] |
10:19 |
<ema> |
depool cp3030 for testing T219967 |
[production] |
09:48 |
<marostegui> |
Remove labtestservices2001 from tendril - T218022 |
[production] |
09:11 |
<moritzm> |
restarting AQS on aqs1004 for glibc update |
[production] |
08:42 |
<elukey> |
restart pdfrender on scb1003 (alert flapping) |
[production] |
08:21 |
<moritzm> |
uploaded php-xdebug 2.7.0+wmf1 for component/php72 (T221923) |
[production] |
07:20 |
<moritzm> |
installing glibc updates on a number of analytics hosts |
[production] |
04:58 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 T221782 (duration: 00m 56s) |
[production] |
00:31 |
<eileen> |
civicrm revision changed from 88736c7c11 to 34027da7df, config revision is 2119df9495 |
[production] |
2019-04-25
§
|
23:41 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
23:41 |
<robh@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
23:41 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
23:41 |
<robh@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
23:39 |
<eileen> |
civicrm revision changed from 519fe8028e to 88736c7c11, config revision is 2119df9495 - deployed patch to start recording payment_processor_id on recurring |
[production] |
22:57 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
22:56 |
<robh@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
22:56 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
22:56 |
<robh@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
21:31 |
<andrewbogott> |
stopping nova services on labnet1001/1002 |
[production] |