701-750 of 10000 results (51ms)
2020-02-03 §
09:32 <elukey> roll restart yarn node managers again to pick up spark encryption/authentication settings [analytics]
08:34 <elukey> stop timers on an-coord1001 to drain the cluster and ease the deploy of spark encryption settings [analytics]
07:58 <elukey> roll restart hadoop yarn node managers to pick up new libcrypto.so link (shouldn't be necessary but just in case) [analytics]
07:24 <elukey> create /usr/lib/x86_64-linux-gnu/libcrypto.so on all the analytics nodes via puppet [analytics]
02:22 <Cam11598> Restarted CVNBot15 and Cubbie [cvn]
02:19 <bd808> Hard restart of webservice to pick up redirect logic (T242719) [tools.docker-registry]
2020-02-02 §
19:25 <effie> restart varnish on cp4028 [production]
17:18 <wm-bot> <lucaswerkmeister> deployed db2a7c16ae (Python 3.7, 2020 Kubernetes cluster), includes venv rebuild; tool had apparently been broken before for about half a month [tools.quickcategories]
08:48 <effie> reboot host analytics1061 - T244081 [production]
06:57 <thcipriani|afk> deployment-deploy01$ while read host; do ssh-keyscan -H "$host"; done < <(grep -vP '^#' /etc/dsh/group/mediawiki-installation | grep -vP '^$' ) | sudo -u jenkins-deploy tee /mnt/home/jenkins-deploy/.ssh/known_hosts [releng]
2020-02-01 §
18:17 <effie> pool scb2003, no need for host to stay depooled - T244069 [production]
17:46 <cdanis> T243634 ✔️ cdanis@cp4030.ulsfo.wmnet ~ 🕐☕ sudo varnish-frontend-restart [production]
17:27 <effie> depool scb2003 T244069 [production]
16:51 <effie> pool mw1273 [production]
16:50 <effie> pool scb2003 [production]
16:30 <elukey> powerup analytics1073 (attempt to see if it was only a kernel-related crash) - T244064 [production]
16:16 <effie> poweroff analytics1073 - T244064 [production]
16:16 <effie> poweroff analytics1073 - /T244064 [production]
16:16 <effie> poweroff analytics1073 [production]
13:00 <effie> depool scb2003 [production]
12:21 <effie> depool mw1273 [production]
01:03 <eileen> process-control config revision is c3c8bde761 [production]
00:50 <eileen> civicrm revision changed from fcc5673ee7 to ee9edf8137, config revision is 2a61da0ace [production]
2020-01-31 §
22:25 <eileen> civicrm revision changed from ac730a6bcb to fcc5673ee7, config revision is 2a61da0ace [production]
22:14 <bstorm_> repooled labsdb1011 now that view work is done [production]
22:00 <eileen> process-control config revision is 2a61da0ace disabled process-control [production]
21:59 <bstorm_> depooled labsdb1011 [production]
21:32 <bstorm_> updated views on labsdb1010 [production]
21:22 <bstorm_> updated views on labsdb1009 [production]
21:21 <bstorm_> updated actor views on labsdb1012 [production]
18:17 <bblack> repool cp4032 (buster) [production]
18:17 <bblack@cumin1001> conftool action : set/pooled=yes; selector: name=cp4032.ulsfo.wmnet [production]
18:14 <bblack> repool cp4029 [production]
18:13 <bblack> restarted ats-tls and varnish-fe on cp4029 [production]
18:05 <bblack> depool varnish-fe on cp4029 [production]
18:03 <bblack> depool ats-tls on cp4029 [production]
16:59 <marostegui> Re-enable notifications on the dbstore1005:3318 check T243871 [production]
16:22 <bd808> Created wikimania-scholarships project and granted qgil and soworu admin rights (T243995) [wikimania-scholarships]
14:05 <arturo> leave tools-prometheus-01 as the backend for tools-prometheus.wmflabs.org for the weekend so grafana dashboards keep working (T238096) [tools]
14:00 <arturo> syncing again prometheus data from tools-prometheus-01 to tools-prometheus-0{3,4} due to some inconsistencies preventing prometheus from starting (T238096) [tools]
11:47 <Lucas_WMDE> lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php enwiki 'Lucas Werkmeister (WMDE)' --force --custom-groups=bureaucrat # make myself bureaucrat to make colleagues sysops [releng]
09:18 <addshore> addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --sleep 4 --batch-size=25 # In a screen for T219301 [production]
03:22 <mutante> powercycling crashed cp3063 [production]
01:09 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@322ee4c]: Update mobileapps to 3eec28d (duration: 06m 53s) [production]
01:02 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@322ee4c]: Update mobileapps to 3eec28d [production]
00:48 <James_F> Restarting Jenkins. [releng]
00:41 <mutante> contint1001/contint2001 - upgrading jenkins to 2.219 [production]
00:36 <mutante> releases2001: upgrading jenkins to 2.219; install1002: import jenkins 2.219 into jessie-wikimedia APT repo [production]
00:31 <mutante> importing jenkins 2.219 to stretch-wikimedia APT repo; releases1001: upgrading jenkins to 2.219 [production]
2020-01-30 §
21:04 <andrewbogott> also apt-get install python3-novaclient on tools-prometheus-03 and tools-prometheus-04 to suppress cronspam. Possible real fix for this is https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/569084/ [tools]