151-200 of 10000 results (27ms)
2016-03-18 §
07:41 <jynus> restarting dbstore2002 to apply new mysql config [production]
06:29 <gehel> restarting elasticsearch server elastic1023.eqiad.wmnet [production]
02:38 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Mar 18 02:38:00 UTC 2016 (duration 8m 40s) [production]
02:29 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.17) (duration: 09m 58s) [production]
01:31 <maxsem@tin> Synchronized php-1.27.0-wmf.17/extensions/CirrusSearch/: Emergency fix https://gerrit.wikimedia.org/r/#/c/278224/ (duration: 00m 36s) [production]
00:38 <ebernhardson> reboot elastic1022.eqiad.wmnet for kernel upgrade [production]
00:09 <ebernhardson> reboot elastic1021.eqiad.wmnet for kernel upgrade [production]
00:09 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/276934/ (duration: 00m 26s) [production]
00:02 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/276410/ (duration: 00m 26s) [production]
00:01 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278206/ (duration: 00m 25s) [production]
2016-03-17 §
23:57 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/276917 (duration: 00m 26s) [production]
23:53 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278021/ (duration: 00m 26s) [production]
23:46 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278029/ (duration: 00m 28s) [production]
23:42 <krenair@tin> Synchronized php-1.27.0-wmf.17/includes/specials/SpecialUploadStash.php: https://gerrit.wikimedia.org/r/#/c/278190/ (duration: 00m 27s) [production]
23:39 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278198/ (duration: 00m 37s) [production]
23:25 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278032/ (duration: 00m 32s) [production]
23:21 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278054/ (duration: 00m 47s) [production]
22:52 <bd808@tin> Synchronized php-1.27.0-wmf.17/includes/api/ApiMain.php: Cast API timeSpentBackend to an int (duration: 00m 25s) [production]
22:47 <gehel> resetting cluster.routing.allocation.disk.watermark.high to 90% on eqiad elasticsearch cluster - shards have moved round, cluster is mostly balanced [production]
22:42 <bd808@tin> Synchronized wmf-config/InitialiseSettings.php: Disable ApiRequest properly (T108618) (duration: 00m 27s) [production]
22:36 <gehel> restarting elasticsearch server elastic1020.eqiad.wmnet [production]
22:17 <ori@tin> Synchronized php-1.27.0-wmf.17/includes/specials/SpecialUploadStash.php: Revert: Debug code for T130204 (it worked!) (duration: 00m 25s) [production]
22:14 <ori@tin> Synchronized php-1.27.0-wmf.17/includes/specials/SpecialUploadStash.php: Debug code for T130204 (duration: 00m 28s) [production]
22:12 <bd808@tin> Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 28s) [production]
22:10 <bd808@tin> Synchronized wmf-config/InitialiseSettings.php: Disable ApiRequest kafka logging (T108618) (duration: 00m 31s) [production]
22:05 <bd808@tin> Synchronized wmf-config/InitialiseSettings.php: Add ApiRequest kafka logging (T108618) (duration: 00m 34s) [production]
22:04 <bd808@tin> Synchronized wmf-config/event-schemas: Add ApiRequest kafka logging (T108618) (duration: 00m 38s) [production]
21:47 <bd808> deleted /var/lib/l10nupdate/caches/cache-1.27.0-wmf.1[345] on tin. Freed ~4G of disk [production]
21:46 <gehel> reducing cluster.routing.allocation.disk.watermark.high to 70% on eqiad elasticsearch cluster [production]
21:07 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: all to 1.27.0-wmf.17 [production]
20:51 <twentyafterfour@tin> Finished scap: sync php-1.27.0-wmf.17 (duration: 26m 57s) [production]
20:40 <urandom> Issuing `nodetool scrub -s -- local_group_wikipedia_T_parsoid_html data` on restbase2004.eqiad.wmnet : T130254 [production]
20:24 <twentyafterfour@tin> Started scap: sync php-1.27.0-wmf.17 [production]
20:14 <urandom> Starting Cassandra on restbase2004.codfw.wmnet (OOM. again.) [production]
20:12 <urandom> Restarting Cassandra on restbase1007-a.eqiad.wmnet (compaction seems stalled) [production]
20:00 <gehel> restarting elasticsearch server elastic1019.eqiad.wmnet [production]
19:41 <gehel> restarting elasticsearch server elastic1018.eqiad.wmnet [production]
18:36 <cscott> updated OCG to version c1a8232594fe846bd2374efd8f7c20d7e97ac449 [production]
18:28 <moritzm> rebooting labnet1001 for kernel update [production]
18:23 <moritzm> rebooting labcontrol1002 for kernel update [production]
18:06 <moritzm> rebooting holmium for kernel update [production]
18:03 <moritzm> rebooting silver for kernel update [production]
18:00 <mobrovac> restbase restarted cassandra on restbase2004, OOM [production]
17:57 <gehel> restarting elasticsearch server elastic1017.eqiad.wmnet [production]
17:10 <urandom> rolling restart of restbase production complete [production]
16:51 <urandom> rolling restart of restbase production to apply https://gerrit.wikimedia.org/r/#/c/277112/ and https://gerrit.wikimedia.org/r/#/c/277836/ [production]
16:45 <urandom> performing restart of restbase1003.eqiad.wmnet (canary) to apply https://gerrit.wikimedia.org/r/#/c/277112/ and https://gerrit.wikimedia.org/r/#/c/277836/ [production]
16:41 <urandom> rolling restart of restbase staging complete [production]
16:41 <gehel> restarting elasticsearch server elastic1016.eqiad.wmnet [production]
16:35 <urandom> performing rolling restart of restbase staging to apply https://gerrit.wikimedia.org/r/#/c/277112/ and https://gerrit.wikimedia.org/r/#/c/277836/ [production]