201-250 of 1366 results (10ms)
2018-02-20 §
12:42 <arturo> upgrading tools-k8s-etcd-01 [tools]
2018-02-19 §
19:13 <arturo> upgrade all packages of tools-services-01 [tools]
19:02 <arturo> move tools-services-01 from puppet3 to puppet4 (manual package upgrade). No issues detected. [tools]
18:23 <arturo> upgrade packages of tools-cron-01 from all channels (trusty-wikimedia, trusty-updates and trusty-tools) [tools]
12:54 <arturo> puppet run with clush to ensure puppet is back to normal after being broken due to duplicated python3 declaration [tools]
2018-02-16 §
18:21 <arturo> upgrading tools-proxy-01 and tools-paws-master-01, same as others [tools]
17:36 <arturo> upgrading oldstable, jessie-backports, jessie-wikimedia packages in tools-k8s-master-01 (excluding linux*, libpam*, nslcd) [tools]
13:00 <arturo> upgrades in tools-exec-14[01-10].eqiad.wmflabs were fine [tools]
12:42 <arturo> aborrero@tools-clushmaster-01:~$ clush -q -w @exec-upgrade-canarys 'DEBIAN_FRONTEND=noninteractive sudo apt-upgrade -u upgrade trusty-updates -y' [tools]
11:58 <arturo> aborrero@tools-elastic-01:~$ sudo apt-upgrade -u -f exclude upgrade jessie-wikimedia -y [tools]
11:57 <arturo> aborrero@tools-elastic-01:~$ sudo apt-upgrade -u -f exclude upgrade jessie-backports -y [tools]
11:53 <arturo> (10 exec canary nodes) aborrero@tools-clushmaster-01:~$ clush -q -w @exec-upgrade-canarys 'sudo apt-upgrade -u upgrade trusty-wikimedia -y' [tools]
11:41 <arturo> aborrero@tools-elastic-01:~$ sudo apt-upgrade -u -f exclude upgrade oldstable -y [tools]
2018-02-15 §
13:54 <arturo> cleanup ferm (deinstall) in tools-services-01 for T187435 [tools]
13:28 <arturo> aborrero@tools-bastion-02:~$ sudo apt-upgrade -u upgrade trusty-tools [tools]
13:16 <arturo> aborrero@tools-bastion-02:~$ sudo apt-upgrade -u upgrade trusty-updates -y [tools]
13:13 <arturo> aborrero@tools-bastion-02:~$ sudo apt-upgrade -u upgrade trusty-wikimedia [tools]
13:06 <arturo> aborrero@tools-webgrid-generic-1401:~$ sudo apt-upgrade -u upgrade trusty-tools [tools]
12:57 <arturo> aborrero@tools-webgrid-generic-1401:~$ sudo apt-upgrade -u upgrade trusty-updates [tools]
12:51 <arturo> aborrero@tools-webgrid-generic-1401:~$ sudo apt-upgrade -u upgrade trusty-wikimedia [tools]
2018-02-14 §
13:09 <arturo> the reboot was OK, the server seems working and kubectl sees all the pods running in the deployment (T187315) [tools]
13:04 <arturo> reboot tools-paws-master-01 for T187315 [tools]
2018-02-11 §
01:28 <zhuyifei1999_> `# find /home/ -maxdepth 1 -perm -o+w \! -uid 0 -exec chmod -v o-w {} \;` Affected: only /home/tr8dr, mode 0777 -> 0775 [tools]
01:21 <zhuyifei1999_> `# find /data/project/ -maxdepth 1 -perm -o+w \! -uid 0 -exec chmod -v o-w {} \;` Affected tools: wikisource-tweets, gsociftttdev, dow, ifttt-testing, elobot. All mode 2777 -> 2775 [tools]
2018-02-09 §
10:35 <arturo> deploy https://gerrit.wikimedia.org/r/#/c/409226/ T179343 T182562 T186846 [tools]
06:15 <bd808> Killed orphan processes owned by iabot, dupdet, and wsexport scattered across the webgrid nodes [tools]
05:07 <bd808> Killed 4 orphan php-fcgi processes from jembot that were running on tools-webgrid-lighttpd-1426 [tools]
05:06 <bd808> Killed 4 orphan php-fcgi processes from jembot that were running on tools-webgrid-lighttpd-1411 [tools]
05:05 <bd808> Killed 1 orphan php-fcgi process from jembot that were running on tools-webgrid-lighttpd-1409 [tools]
05:02 <bd808> Killed 4 orphan php-fcgi processes from jembot that were running on tools-webgrid-lighttpd-1421 and pegging the cpu there [tools]
04:56 <bd808> Rescheduled 30 of the 60 tools running on tools-webgrid-lighttpd-1421 (T186830) [tools]
04:39 <bd808> Killed 4 orphan php-fcgi processes from jembot that were running on tools-webgrid-lighttpd-1417 and pegging the cpu there [tools]
2018-02-08 §
18:38 <arturo> aborrero@tools-k8s-master-01:~$ sudo kubectl uncordon tools-worker-1002.tools.eqiad.wmflabs [tools]
18:35 <arturo> aborrero@tools-worker-1002:~$ sudo apt-upgrade -u upgrade jessie-wikimedia -v [tools]
18:33 <arturo> aborrero@tools-worker-1002:~$ sudo apt-upgrade -u upgrade oldstable -v [tools]
18:28 <arturo> cordon & drain tools-worker-1002.tools.eqiad.wmflabs [tools]
18:10 <arturo> uncordon tools-paws-worker-1019. Package upgrades were OK. [tools]
18:08 <arturo> aborrero@tools-paws-worker-1019:~$ sudo apt-upgrade upgrade stable -v [tools]
18:06 <arturo> aborrero@tools-paws-worker-1019:~$ sudo apt-upgrade upgrade stretch-wikimedia -v [tools]
18:02 <arturo> cordon tools-paws-worker-1019 to do some package upgrades [tools]
17:29 <arturo> repool tools-exec-1401.tools.eqiad.wmflabs. Package upgrades were OK. [tools]
17:20 <arturo> aborrero@tools-exec-1401:~$ sudo apt-upgrade upgrade trusty-updates -vy [tools]
17:15 <arturo> aborrero@tools-exec-1401:~$ sudo apt-upgrade upgrade trusty-wikimedia -vy [tools]
17:11 <arturo> depool tools-exec-1401.tools.eqiad.wmflabs to do some package upgrades [tools]
14:22 <arturo> it was some kind of transient error. After a second puppet run across the fleet, all seems fine [tools]
13:53 <arturo> deploy https://gerrit.wikimedia.org/r/#/c/407465/ which is causing some puppet issues. Investigating. [tools]
2018-02-06 §
13:15 <arturo> deploy https://gerrit.wikimedia.org/r/#/c/408529/ to tools-services-01 [tools]
13:05 <arturo> unpublish/publish trusty-tools repo [tools]
13:03 <arturo> install aptly v0.9.6-1 in tools-services-01 for T186539 after adding it to trusty-tools repo (self contained) [tools]
2018-02-05 §
17:58 <arturo> publishing/unpublishing trusty-tools repo in tools-services-01 to address T186539 [tools]