6351-6400 of 10000 results (49ms)
2017-02-02 ยง
16:13 <elukey> rebooting mc202[6789] (not serving any traffic) to see if they come up cleanly [production]
16:00 <elukey> rebooting mc203[01234] (not serving any traffic) to see if they come up cleanly [production]
15:43 <moritzm> upgrading firejail on remaining app servers [production]
15:35 <moritzm> upgrading firejail on wtp1001 [production]
15:15 <moritzm> upgrading firejail on eqiad imagescalers [production]
15:11 <elukey> rebooting mc203[56] (not taking any traffic) to test if they come up cleanly [production]
15:01 <elukey> Replace Redis/Memcached shards mc200[4567] with mc202[2345] [production]
14:51 <godog> manually fail sdc on graphite1001 - T157022 [production]
14:23 <addshore> EU SWAT really finished [production]
14:23 <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT [[gerrit:334997|Create Wikiprojekti namespace on Finnish Wikipedia]] T156621 (duration: 00m 41s) [production]
14:14 <addshore> EU SWAT finished [production]
14:14 <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT [[gerrit:335481|Add Wikinews languages (en, pt, ca, fr, de, it) as import sources on eswikinews]] T156737 (duration: 00m 40s) [production]
14:10 <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT [[gerrit:324489|Enable ElectronPdfService extension on dewiki]] T150942 (duration: 00m 41s) [production]
12:53 <gehel> starting reimage to jessie of elasticsearch relforge - T151326 [production]
11:59 <joal@tin> Finished deploy [analytics/refinery@bc4b4ed]: (no justification provided) (duration: 01m 14s) [production]
11:58 <joal@tin> Started deploy [analytics/refinery@bc4b4ed]: (no justification provided) [production]
11:40 <elukey> Swap mc2002 with mc2020, mc2003 with mc2021 (Redis codfw replicas) - T155755 [production]
11:35 <joal@tin> Finished deploy [analytics/refinery@bc4b4ed]: (no justification provided) (duration: 03m 03s) [production]
11:32 <joal@tin> Started deploy [analytics/refinery@bc4b4ed]: (no justification provided) [production]
10:53 <elukey> Swap mc2001 with mc2019 (Redis codfw replicas) - T155755 [production]
10:34 <moritzm> restarted hadoop-mapreduce-historyserver on analytics1001 [production]
10:23 <moritzm> rolling restart of nginx tls terminators running on mw* application servers in eqiad to pick up openssl 1.1 update [production]
09:54 <moritzm> rolling restart of logstash cluster to pick up openjdk/NSS security updates [production]
09:19 <jynus> deploying schema change to page_assessments_projects on enwikivoyage T156305 [production]
09:18 <moritzm> uograding remaining canary servers to new HHVM packages [production]
09:17 <marostegui> Remove dbstore1001:/srv/tmp/db1063.tar.gz after it has been transferred to db1095:/srv/tmp/db1063.tar.gz to get more disk space [production]
08:57 <jynus> deploying schema change to page_assessments_projects on enwiki T156305 [production]
08:42 <filippo@tin> Finished deploy [prometheus/jmx_exporter@23a8f0b]: jmx_exporter deploy (duration: 00m 04s) [production]
08:42 <filippo@tin> Started deploy [prometheus/jmx_exporter@23a8f0b]: jmx_exporter deploy [production]
08:30 <moritzm> installing ntfs-3g security update on labnodepool (other servers had it deinstalled) [production]
08:26 <filippo@tin> Finished deploy [prometheus/jmx_exporter@23a8f0b]: (no justification provided) (duration: 00m 07s) [production]
08:26 <filippo@tin> Started deploy [prometheus/jmx_exporter@23a8f0b]: (no justification provided) [production]
08:24 <marostegui> Transfer /srv/tmp/db1063.tar.gz from dbstore1001 to db1095:/srv/tmp to gain disk space [production]
08:24 <marostegui> Remove /srv/tmp/db1067.tar.gz from dbstore1001 to gain disk space [production]
08:23 <filippo@tin> Finished deploy [prometheus/jmx_exporter@23a8f0b]: (no justification provided) (duration: 00m 12s) [production]
08:23 <filippo@tin> Started deploy [prometheus/jmx_exporter@23a8f0b]: (no justification provided) [production]
08:10 <legoktm@tin> Synchronized php-1.29.0-wmf.10/includes/specials/pagers/ActiveUsersPager.php: Make last remaining user_groups queries honor $wgDisableUserGroupExpiry https://gerrit.wikimedia.org/r/#/c/335587/ (T156995) (duration: 00m 51s) [production]
08:08 <legoktm@tin> Synchronized php-1.29.0-wmf.10/includes/api/ApiQueryAllUsers.php: Make last remaining user_groups queries honor $wgDisableUserGroupExpiry https://gerrit.wikimedia.org/r/#/c/335587/ (T156995) (duration: 00m 58s) [production]
07:41 <marostegui> Restart MySQL on db2012 to tune some innodb_ft flags - T156905 [production]
03:24 <eileen1> renable all other jenkins jobs - only some dedupe & one-off jobs disabled [production]
03:17 <eileen1> re-enable dedupe jobs [production]
03:14 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Feb 2 03:14:06 UTC 2017 (duration 5m 44s) [production]
03:08 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.10) (duration: 13m 49s) [production]
02:36 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.9) (duration: 14m 39s) [production]
02:32 <eileen1> update civicrm from d06db9223a47f74f9b87eba0f008696489668c77 to e45da6dc2cfaa48da8807ab9a571810c119e3233 [production]
02:23 <eileen1> drush dis -y module_missing_message_fixer [production]
02:22 <eileen1> drush mmmff --all [production]
02:22 <eileen1> run drush en -y module_missing_message_fixer [production]
02:20 <eileen1> Update civicrm from 7a861216a1392bf157fc4f17fd9e497a3796c00e to d06db9223a47f74f9b87eba0f008696489668c77 [production]
02:02 <mutante> carbon - remove unmapped IPv6 address making ferm rules fail, use only the _mapped_ IP (ip addr del 2620:0:861:1:7a2b:cbff:fe09:ea0/64 dev eth0) (T84380 T132757) [production]