2051-2100 of 10000 results (70ms)
2019-04-28 §
14:47 <jforrester@deploy1001> Synchronized static/images/project-logos/trwiki.png: trwiki: Update logo for 2 year anniversary, part III (duration: 00m 53s) [production]
14:45 <jforrester@deploy1001> Synchronized static/images/project-logos/trwiki-1.5x.png: trwiki: Update logo for 2 year anniversary, part II (duration: 00m 53s) [production]
14:44 <jforrester@deploy1001> Synchronized static/images/project-logos/trwiki-2x.png: trwiki: Update logo for 2 year anniversary, part I (duration: 00m 55s) [production]
2019-04-27 §
17:44 <elukey> restart pdfrender on scb1002 (alert flapping) [production]
12:37 <jynus> correcting last log, stopping dbstore2002:s1 to clone it to db2097 T220572 [production]
12:37 <jynus> stopping dbstore2002:s6 to clone it to db2097 T220572 [production]
00:11 <foks> reset passwords for FritzSolms@global and Seanhood@global [production]
2019-04-26 §
20:15 <foks> changing email and password for "Lemon martini@global" [production]
19:38 <foks> changing password for JDiPierro@global [production]
19:21 <bblack> varnish-backend-restart on cp4026, evidence of artificial 503s from mbox lag behavior, probably related to the semi-abuse client doing odd 404 traffic to ulsfo that's triggering bugs in swift's rewrite.py .... [production]
19:04 <foks> changing password for Subinsebastien [production]
17:50 <mutante> analytics1052 - reported broken systemd state in Icinga - service mcelog was in state failed - systemctl start mcelog - (T212219 ?) [production]
16:18 <jynus> stop s6 mariadb instance on dbstore2001 T220572 [production]
15:34 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: thumbor1001 ms-fe1005 ms-be1013 scb1001 restbase1007 [production]
15:05 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: ores1001.yaml wtp1025.yaml rdb1006.yaml [production]
14:18 <marostegui> Set pc1004-1006 and pc2004-2006 as unracked on netbox - T209858 T210969 [production]
13:17 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: mw1311.yaml, mx2001 & dubnium [production]
12:52 <ema> cp4025: restart varnish-be due to mbox lag [production]
12:50 <jijiki> Restarting hhvm on mw1288 [production]
12:48 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on mc1019, maps1001 and logstash1007 [production]
12:45 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: cluster=cache_upload,name=cp4021.ulsfo.wmnet,dc=ulsfo [production]
12:44 <ema> pool cp4021 w/ ATS backend T219967 [production]
12:20 <ema> repool cp3030 after directors.frontend.vcl testing T219967 [production]
12:09 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on canary hosts: elastic1017, ganeti2001, analytics1042 [production]
11:26 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on lvs4007, dns2001 and multatuli [production]
11:16 <jbond42> upgrade puppet 4=> 5 and facter 2 => 3 on bast4002, aqs1004 and conf2001 [production]
10:28 <moritzm> restarting Parsoid on wtp1025 for glibc update [production]
10:19 <ema> depool cp3030 for testing T219967 [production]
09:48 <marostegui> Remove labtestservices2001 from tendril - T218022 [production]
09:11 <moritzm> restarting AQS on aqs1004 for glibc update [production]
08:42 <elukey> restart pdfrender on scb1003 (alert flapping) [production]
08:21 <moritzm> uploaded php-xdebug 2.7.0+wmf1 for component/php72 (T221923) [production]
07:20 <moritzm> installing glibc updates on a number of analytics hosts [production]
04:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 T221782 (duration: 00m 56s) [production]
00:31 <eileen> civicrm revision changed from 88736c7c11 to 34027da7df, config revision is 2119df9495 [production]
2019-04-25 §
23:41 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:41 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
23:41 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:41 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
23:39 <eileen> civicrm revision changed from 519fe8028e to 88736c7c11, config revision is 2119df9495 - deployed patch to start recording payment_processor_id on recurring [production]
22:57 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
22:56 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
22:56 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
22:56 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
21:31 <andrewbogott> stopping nova services on labnet1001/1002 [production]
21:26 <andrewbogott> revoking M5 grants as per https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/506428/4/modules/role/templates/mariadb/grants/production-m5.sql.erb and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/506345/3/modules/role/templates/mariadb/grants/production-m5.sql.erb [production]
21:12 <tgr> T221516 running mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=metawiki --logwiki=metawiki 'FoldDownPro' 'MichaelOBFDP' [production]
19:45 <mobrovac@deploy1001> Finished deploy [restbase/deploy@7187e0c]: Bump HTML content version in docs, remove Parsoid stash fall-back and start logging all sections requests - T221432 T215956 T216636 (duration: 20m 04s) [production]
19:25 <mobrovac@deploy1001> Started deploy [restbase/deploy@7187e0c]: Bump HTML content version in docs, remove Parsoid stash fall-back and start logging all sections requests - T221432 T215956 T216636 [production]
19:24 <mobrovac@deploy1001> Finished deploy [restbase/deploy@7187e0c] (dev-cluster): Bump HTML content version in docs and remove Parsoid stash fall-back (duration: 03m 10s) [production]