2151-2200 of 10000 results (64ms)
2020-02-03 §
11:38 <ema> powercycle cp3057 T244127 T238305 [production]
10:24 <godog> temp disable puppet on cp hosts as precaution for https://gerrit.wikimedia.org/r/c/operations/puppet/+/563977 [production]
10:08 <moritzm> installing sudo security updates on stretch [production]
2020-02-02 §
19:25 <effie> restart varnish on cp4028 [production]
08:48 <effie> reboot host analytics1061 - T244081 [production]
2020-02-01 §
18:17 <effie> pool scb2003, no need for host to stay depooled - T244069 [production]
17:46 <cdanis> T243634 ✔️ cdanis@cp4030.ulsfo.wmnet ~ 🕐☕ sudo varnish-frontend-restart [production]
17:27 <effie> depool scb2003 T244069 [production]
16:51 <effie> pool mw1273 [production]
16:50 <effie> pool scb2003 [production]
16:30 <elukey> powerup analytics1073 (attempt to see if it was only a kernel-related crash) - T244064 [production]
16:16 <effie> poweroff analytics1073 - T244064 [production]
16:16 <effie> poweroff analytics1073 - /T244064 [production]
16:16 <effie> poweroff analytics1073 [production]
13:00 <effie> depool scb2003 [production]
12:21 <effie> depool mw1273 [production]
01:03 <eileen> process-control config revision is c3c8bde761 [production]
00:50 <eileen> civicrm revision changed from fcc5673ee7 to ee9edf8137, config revision is 2a61da0ace [production]
2020-01-31 §
22:25 <eileen> civicrm revision changed from ac730a6bcb to fcc5673ee7, config revision is 2a61da0ace [production]
22:14 <bstorm_> repooled labsdb1011 now that view work is done [production]
22:00 <eileen> process-control config revision is 2a61da0ace disabled process-control [production]
21:59 <bstorm_> depooled labsdb1011 [production]
21:32 <bstorm_> updated views on labsdb1010 [production]
21:22 <bstorm_> updated views on labsdb1009 [production]
21:21 <bstorm_> updated actor views on labsdb1012 [production]
18:17 <bblack> repool cp4032 (buster) [production]
18:17 <bblack@cumin1001> conftool action : set/pooled=yes; selector: name=cp4032.ulsfo.wmnet [production]
18:14 <bblack> repool cp4029 [production]
18:13 <bblack> restarted ats-tls and varnish-fe on cp4029 [production]
18:05 <bblack> depool varnish-fe on cp4029 [production]
18:03 <bblack> depool ats-tls on cp4029 [production]
16:59 <marostegui> Re-enable notifications on the dbstore1005:3318 check T243871 [production]
09:18 <addshore> addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --sleep 4 --batch-size=25 # In a screen for T219301 [production]
03:22 <mutante> powercycling crashed cp3063 [production]
01:09 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@322ee4c]: Update mobileapps to 3eec28d (duration: 06m 53s) [production]
01:02 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@322ee4c]: Update mobileapps to 3eec28d [production]
00:41 <mutante> contint1001/contint2001 - upgrading jenkins to 2.219 [production]
00:36 <mutante> releases2001: upgrading jenkins to 2.219; install1002: import jenkins 2.219 into jessie-wikimedia APT repo [production]
00:31 <mutante> importing jenkins 2.219 to stretch-wikimedia APT repo; releases1001: upgrading jenkins to 2.219 [production]
2020-01-30 §
19:37 <mutante> copying /var/log/apache2 to /root on all eqiad mw appservers to preserve logs [production]
18:07 <vgutierrez> depool cp4032 and perform a rolling restart of varnish-fe at cp4027-cp4031 - T243634 [production]
17:51 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.16/extensions/Wikibase/lib/includes/Store/Sql/Terms/FingerprintableEntityTermStoreTrait.php: wbterms: Fix incorrect deletion of rows in findActuallyUnusedTermIds (T243944) (duration: 01m 06s) [production]
17:49 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.16/extensions/Wikibase/repo/maintenance/rebuildItemTerms.php: wbterms: Write only to the new term store in rebuildItemTerms (T243944) (duration: 01m 09s) [production]
17:03 <vgutierrez> repooling cp4032 - T243634 [production]
17:02 <vgutierrez> restarting varnish-frontend on cp4031 before it crashes - T243634 [production]
16:26 <vgutierrez> manually refreshing OCSP stapling response for non-canonical-redirects-3 - T243948 [production]
12:22 <arturo> add prometheus 2.7.1+ds-3+k8s+buster to buster-wikimedia T238096 (basically a rebuild from stretch) [production]
06:23 <vgutierrez> restarting varnish-frontend on cp4030 before it crashes - T243634 [production]
06:21 <vgutierrez> depool cp4032 - T243634 [production]
05:12 <vgutierrez> restarting varnish-frontend and repooling cp4029 - T243634 [production]