401-450 of 10000 results (29ms)
2016-02-25 §
16:23 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: SWAT: wikidatawiki to wmf.14 [production]
16:16 <elukey> added mc1014 back to the redis/memcached pool after maintenance [production]
16:16 <thcipriani@tin> Synchronized php-1.27.0-wmf.14/resources/lib/oojs-ui/oojs-ui-core.js: SWAT: OOjs UI: Fix #gatherPreInfuseState called incorrectly, causing TypeErrors [[gerrit:273250]] (duration: 01m 42s) [production]
15:32 <godog> stop cassandra/restbase on restbase2001 to finish raid0 grow [production]
15:21 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: wikidata back to wmf.13 for now [production]
11:47 <hashar> Reverting session manager cherry picks from wmf branches ( https://gerrit.wikimedia.org/r/#/c/273201/ and https://gerrit.wikimedia.org/r/#/c/273202/ ) they have not been deployed after they got merged [production]
11:33 <godog> depool ms-fe1004 for trusty dist-upgrade [production]
11:23 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1021 and db1024 (duration: 01m 45s) [production]
10:58 <moritzm> powercycling cp2010 [production]
10:41 <godog> set xff to 0.01 for graphite metrics swift.*.containers (was 0.5) [production]
10:03 <hashar> starting Jenkins [production]
09:57 <hashar> Stopping Jenkins [production]
09:29 <elukey> removed mc1014.eqiad from the redis/memcached pool for maintenance [production]
04:44 <urandom> decommissioning Cassandra on restbase1008-a.eqiad.wmnet T119935 [production]
04:35 <urandom> restarting restbase1008-a to cancel rebuild T108611 T119935 [production]
03:19 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Feb 25 03:19:30 UTC 2016 (duration 9m 6s) [production]
03:10 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.14) (duration: 18m 00s) [production]
02:35 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.13) (duration: 19m 03s) [production]
00:57 <bd808> Started crashed Logstash process on logstash1002 (systemd doesn't restart authomatically due to T127677) [production]
00:03 <ori@tin> Synchronized wmf-config/CommonSettings.php: I4cc836f3ca: Fully-qualify EventLoggingBaseUri (duration: 01m 40s) [production]
00:01 <ori@tin> Synchronized wmf-config/StartProfiler.php: I016e23d81: xhgui: Sample fewer requests (1:100k instead of 1:10k) (duration: 01m 58s) [production]
2016-02-24 §
22:49 <gehel> reboot logstash1005 for kernel and elasticsearch update [production]
22:29 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.14 too [production]
22:09 <gehel> reboot logstash1006 for kernel and elasticsearch update [production]
22:07 <demon@tin> Finished scap: group0 to wmf.14 (duration: 47m 50s) [production]
21:19 <demon@tin> Started scap: group0 to wmf.14 [production]
21:19 <subbu> finished deploying parsoid version 581a43c75 [production]
21:08 <subbu> synced code; restarted parsoid on wtp1001 as a canary [production]
21:01 <subbu> starting parsoid deploy [production]
20:49 <moritzm> reboot logstash1004 for kernel/elasticsearch update [production]
20:39 <gehel> reboot logstash1003 for kernel and elasticsearch update [production]
20:28 <gehel> reboot logstash1002 for kernel and elasticsearch update [production]
20:15 <chasemp> reboot labstore1002 to ensure io scheduler grub options work [production]
20:13 <moritzm> reboot logstash1001 for kernel update [production]
19:46 <chasemp> runonce apply for https://gerrit.wikimedia.org/r/#/c/272891/ for labs vm's (only affects nfs clients) [production]
19:46 <legoktm@tin> Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/273032 (duration: 01m 41s) [production]
19:41 <cmjohnson1> db1021 replacing disk 8 [production]
19:04 <legoktm@tin> Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/273017 (duration: 01m 37s) [production]
18:52 <papaul> es201[1-9] -signing puppet certs, salt-key. initial run [production]
18:39 <mutante> restart gitblit [production]
18:10 <bblack> disabling nginx keepalives on remaining clusters (upload, misc, maps) [production]
18:07 <ori> hafnium did not have enough disk space for mongo to execute db.repairDatabase(), which is necessary for reclaiming disk space. Since existing profile data can be tossed, ran `db.dropDatabase(); db.repairDatabase();`. Need to think this through better, obviously. [production]
18:02 <ori> mongodb on hafnium: ran `db.results.remove( { "meta.SERVER.REQUEST_URI": "/wiki/Special:BlankPage" } ); db.repairDatabase();` to drop profiles of PyBal requests and compact the database. [production]
17:44 <demon@tin> Synchronized wmf-config/: poolcounter config simplification (duration: 01m 39s) [production]
17:21 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: Re-apply "Set $wgResourceBasePath to /w for medium wikis" (duration: 01m 42s) [production]
17:16 <demon@tin> Synchronized wmf-config/: service entries for initialisesettings + fix (duration: 01m 45s) [production]
16:59 <papaul> es201[1-9] disabling /revoking puppet and salt keys for re-image [production]
16:57 <papaul> es200[1-9] disabling /revoking puppet and salt keys for re-image [production]
16:53 <bd808> https://wmflabs.org/sal/production missing SAL data since 2016-02-21T14:39 due to bot crash; needs to be backfilled from wikitech data [production]
16:43 <hashar> sal on elastic search is stall https://phabricator.wikimedia.org/T127981 [production]