1751-1800 of 10000 results (56ms)
2017-11-20 §
08:46 <marostegui> Run mydumper for db1047.staging - T156844 [production]
07:58 <moritzm> installing procmail security updates [production]
07:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 - T178359 (duration: 00m 49s) [production]
06:41 <marostegui> Stop MySQL on db1082 to clone db1109 and db1110 - T180700 [production]
06:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1082 - T177208 (duration: 00m 48s) [production]
06:25 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore original weights for db1100 and db1071 - T180917 (duration: 00m 49s) [production]
06:15 <marostegui> Reboot db2068 - T180927 [production]
02:28 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.7) (duration: 06m 26s) [production]
00:52 <andrewbogott> cherry-picking https://gerrit.wikimedia.org/r/#/c/392172/ onto the tools puppetmaster [tools]
2017-11-19 §
23:20 <Jamesofur> removed 2FA for Ask21 T180889 [production]
20:28 <krinkle@tin> Synchronized docroot/noc/index.html: noc: Link to Grafana (duration: 00m 49s) [production]
19:50 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1100 (duration: 00m 49s) [production]
13:30 <paladox> upgrading icinga2 to 2.8.0 [git]
07:09 <mobrovac@tin> Started restart [zotero/translators@a0c41c3]: Zotero eating up memory [production]
07:01 <mobrovac@tin> Started restart [electron-render/deploy@8dd5f13]: electron stuck - T174916 [production]
2017-11-18 §
23:25 <madhuvishy> killed pid 25631 - /usr/lib/gridengine/qacct -j 2067313 [tools.ukbot]
21:40 <Krinkle> Deleting cvn-apache8 (replaced by cvn-apache9) [cvn]
20:06 <gehel> upgrade of elasticsearch eqiad complete - T178411 [production]
18:03 <reedy@tin> Synchronized multiversion/vendor/: bump (duration: 01m 14s) [production]
15:53 <marostegui> Compress s5 on db2085 - T178359 [production]
14:48 <marostegui> Compress s3 on db2092 - T178359 [production]
02:36 <Krinkle> Shutdown now-unused cvn-apache8 instance. To be deleted later. [cvn]
02:36 <Krinkle> Switch cvn.wmflabs.org web proxy over to new cvn-apache9 backend. [cvn]
02:14 <Krinkle> Create and provision cvn-apache9 (Debian 9 Stretch) to replace cvn-apache8 (Ubuntu 14 Trusty) [cvn]
02:11 <zhuyifei1999_> Removing myself from maintainer list (cc T179599) [tools.tb-dev]
00:30 <urandom> Bootstrapping restbase2006-b (T179422) [production]
2017-11-17 §
23:17 <Sagan> CAC: Updating packages and running autoremove [rcm]
23:15 <Sagan> CAC: Fixing errors from last ran via foreachwiki update.php --quick [rcm]
23:15 <Sagan> CAC: Running composer update on vagrant [rcm]
23:12 <Sagan> Xenon: Doing package updates and autoremove [rcm]
23:12 <Sagan> Neon: Updating packages [rcm]
23:11 <Sagan> CAC: Updating vagrant, including vagrant git-update [rcm]
23:10 <Sagan> Oxygen: Updateing packages [rcm]
23:08 <Sagan> Tin: Updating packages [rcm]
23:06 <Sagan> Xenon: Running update [rcm]
21:33 <valhallasw`cloud> also g-w'ed those files, and sent emails to all the affected users [tools]
21:31 <JeanFred> Reverted to old database replicas (via `git reset HEAD~1 && git stash`) as part of T180833 investigation [tools.heritage]
21:17 <valhallasw`cloud> chmod o-w'ed a bunch of files reported by Dispenser; writing emails to the owners about this [tools]
20:35 <mutante> netmon1002 - rsync smokeping data back from local backup to show measurements made from eqiad as requested on T180812 [production]
20:05 <mutante> running puppet on all cache misc to switch smokeping web to eqiad [production]
20:02 <mutante> T180812 copying smokeping data from 2001 to 1002 - netmon1002: /usr/bin/rsync -avp rsync://netmon2001.wikimedia.org/var-lib-smokeping /var/lib/smokeping/ | switching backend from codfw to eqiad [production]
19:37 <mutante> T180812 - @netmon1002:# rsync -avp /var/lib/smokeping/ /root/backup/netmon1002/201711717/var/lib/smokeping/@netmon2001:/# rsync -avp /var/lib/smokeping/ /root/backup/netmon2001/201711717/var/lib/smokeping/ [production]
19:36 <mutante> @netmon1002:/var/lib/smokeping# rsync -avp /var/lib/smokeping/ /root/backup/netmon1002/201711717/var/lib/smokeping/ [production]
19:30 <JeanFred> Started a new harvest to better investigate T180833 [tools.heritage]
18:37 <anomie@tin> Synchronized wmf-config/InitialiseSettings.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster, no prod change (duration: 00m 48s) [production]
18:36 <anomie@tin> Synchronized wmf-config/InitialiseSettings-labs.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster, no prod change (duration: 00m 49s) [production]
17:43 <demon@tin> Synchronized php-1.31.0-wmf.8/extensions/Wikidata/extensions/Wikibase/client/WikibaseClient.php: fix client dependencies (duration: 00m 49s) [production]
17:41 <demon@tin> Synchronized php-1.31.0-wmf.8/extensions/Wikibase/client/WikibaseClient.php: fix client dependencies (duration: 00m 50s) [production]
17:21 <bblack> enabling new normalization code for all upload@esams (done) [production]
17:19 <bblack> enabling new normalization code for all upload@eqiad [production]