3901-3950 of 10000 results (37ms)
2017-10-09 §
07:15 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add db2079 to the config - T170662 (duration: 00m 47s) [production]
07:01 <ema> cp4022 - backend restart, mailbox lag, cache_upload [production]
06:56 <marostegui> Stop MySQL on db2075 to use it to clone db2079 - T170662 [production]
06:56 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2075 - T170662 (duration: 00m 46s) [production]
06:25 <marostegui> Drop moodbar_feedback and moodbar_feedback_response from s6 - T153033 [production]
06:14 <marostegui> Optimize pagelinks and templatelinks on db1039 - T174509 [production]
06:11 <marostegui> Optimize pagelinks and templatelinks on db1050 - T174509 [production]
06:08 <marostegui> Optimize pagelinks and templatelinks on db1096 - T174509 [production]
06:06 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1096 - T174509 (duration: 00m 47s) [production]
05:59 <marostegui> Optimize pagelinks and templatelinks on db1091 - T174509 [production]
05:59 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1091 - T174509 (duration: 00m 47s) [production]
05:53 <marostegui> Optimize pagelinks and templatelinks on db1090 - T174509 [production]
05:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1090 - T174509 (duration: 00m 47s) [production]
05:44 <marostegui> Optimize pagelinks and templatelinks on db1083 - T174509 [production]
05:38 <marostegui> Stop MySQL on db1083 to upgrade it [production]
05:37 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1083 - T174509 (duration: 00m 47s) [production]
02:48 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Oct 9 02:48:21 UTC 2017 (duration 6m 54s) [production]
02:41 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.2) (duration: 13m 49s) [production]
2017-10-08 §
21:40 <hoo> Killed "dumpwikidatardf.sh truthy nt" (wikidata truthy dump) on snapshot1007, got stuck after T169680. [production]
11:28 <volans> ack-ed puppet not running on stat100[5-6],snapshot100[1,5-7] due to NFSD stuck on dataset1001 - T169680 [production]
08:19 <elukey> restart varnish backend on cp4026 to stop 503s [production]
2017-10-07 §
22:46 <reedy@tin> Synchronized php-1.31.0-wmf.2/resources/: T177705 (duration: 00m 48s) [production]
22:44 <reedy@tin> Synchronized php-1.31.0-wmf.2/includes/specials/SpecialBlock.php: T177705 (duration: 00m 49s) [production]
2017-10-06 §
18:58 <ejegg> updated CiviCRM from 197b22dce5a6c60a6e3697f5b9ba49de536d0bbb to 276cbfd4ebbf4b1630ab9eb87fe5ea365d624767 [production]
18:38 <catrope@tin> Synchronized php-1.31.0-wmf.2/extensions/TwoColConflict/: Unbreak preference changes (T177524) (duration: 00m 47s) [production]
18:23 <arlolra@tin> Finished deploy [parsoid/deploy@e437c93]: Updating Parsoid to 772e11bf (duration: 12m 41s) [production]
18:10 <arlolra@tin> Started deploy [parsoid/deploy@e437c93]: Updating Parsoid to 772e11bf [production]
15:30 <moritzm> repooling mw2145 (API server). it was previously depooled, but there's no tickets or SAL entries indicating hardware maintenance, so it was probably simply forgotten to repool it [production]
15:25 <moritzm> repooling mw2203 (API server). it was previously depooled, but there's no tickets or SAL entries indicating hardware maintenance, so it was probably simply forgotten to repool it [production]
12:54 <moritzm> repooling mw2248-mw2250 (job runners). they were previosly depooled, but there's no tickets or SAL entries indicating hardware maintenance, so they were probably simply forgotten to repool [production]
11:46 <bblack> cp1* (eqiad caches) - upgrade to nginx-1.13.5-1+wmf1~jessie1 to match all the other sites [production]
11:34 <moritzm> repooling mw2244/mw2245 (image scalers). they were previosly depooled, but there's no tickets or SAL entries indicating hardware maintenance, so they were probably simply forgotten to repool [production]
11:21 <moritzm> installing gdk-pixbuf security updates [production]
11:10 <elukey> rolling restart of all the druid daemons on druid100[1-6] to pick up new logging changes [production]
11:03 <moritzm> installing git security updates on trusty (Debian already fixed) [production]
10:51 <moritzm> upgrading HHVM on script runners to HHVM 3.18.5 [production]
10:47 <moritzm> upgrade remaining video scalers in codfw to HHVM 3.18.5 [production]
10:43 <elukey> restart replication (mysql+ eventlogging_sync) on dbstore1002 after mysql restart [production]
10:17 <moritzm> upgrade remaining job runners in codfw to HHVM 3.18.5 [production]
09:53 <elukey> stop replication (eventlogging_sync + mysql) on dbstore1002 as prep step for mysql restart [production]
09:17 <hashar> Restarted the CI Jenkins for plugin upgrades [production]
09:01 <moritzm> upgrade remaining app servers in codfw to HHVM 3.18.5 [production]
08:40 <moritzm> upgrade remaining image scalers in codfw to HHVM 3.18.5 [production]
08:09 <ema> upgrade pybal to 1.14.0 on eqiad primary LVSs [production]
07:48 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1076 original weight - T174054 (duration: 00m 48s) [production]
07:23 <ema> upgrade pybal to 1.14.0 on eqiad secondary LVSs [production]
07:18 <moritzm> upgrade remaining API servers in codfw to HHVM 3.18.5 [production]
07:11 <legoktm> live hacking on mwdebug1002 [production]
07:05 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight - T174054 (duration: 00m 47s) [production]
06:37 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight - T174054 (duration: 00m 47s) [production]