2851-2900 of 10000 results (37ms)
2016-01-29 §
15:32 <akosiaris> remove all networking configuration from asw-b-eqiad switch for nas1001-a, nas1001-b. Leave just descriptions [production]
15:21 <bblack> upgrading packages (incl kernel) on esams cache hosts (cp3xxx) (codfw, ulsfo already done) [production]
15:11 <akosiaris> powering off nas1001-a.eqiad.wmnet. https://phabricator.wikimedia.org/T124156 [production]
15:08 <akosiaris> powering off nas1001-b.eqiad.wmnet. https://phabricator.wikimedia.org/T124156 [production]
15:01 <elukey> re-enabled puppet on analytics1027 [production]
14:39 <elukey> stopped kafka (service) on kafka1012 (the host that caused the outage) [production]
14:24 <moritzm> rebooting bohrium for kernel update [production]
14:03 <_joe_> installing the new hhvm package on all the codfw appserver [production]
13:43 <_joe_> installing the new HHVM package to the canary appservers (main and api) [production]
12:30 <paravoid> force-rebooting pollux [production]
11:43 <_joe_> uploaded hhvm_3.6.5+dfsg1-1+wm8 to trusty-wikimedia [production]
11:22 <moritzm> rolling restart of swift in codfw [production]
11:14 <elukey> disabled puppet on analytics1027 due to issues with Camus and HDFS [production]
10:17 <moritzm> rolling restart of swift in esams [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Jan 29 02:32:56 UTC 2016 (duration 7m 28s) [production]
02:25 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.11) (duration: 10m 40s) [production]
01:31 <ori@mira> Synchronized wmf-config: I83da57cf: Enable persistent redis connections for job runners (duration: 01m 11s) [production]
01:03 <krenair@mira> Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/267186/ (duration: 01m 09s) [production]
01:01 <krenair@mira> Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/265292/ (duration: 01m 14s) [production]
00:57 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/267071/ (duration: 01m 11s) [production]
00:53 <krenair@mira> Synchronized wmf-config/CirrusSearch-production.php: https://gerrit.wikimedia.org/r/#/c/266995/ (duration: 01m 11s) [production]
00:50 <yurik> synced latest graphoid [production]
00:49 <krenair@mira> Synchronized php-1.27.0-wmf.11/extensions/MobileFrontend/resources/skins.minerva.editor/init.js: https://gerrit.wikimedia.org/r/#/c/267168/ (duration: 01m 12s) [production]
00:45 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/267053/ (duration: 01m 10s) [production]
00:43 <krenair@mira> Synchronized wmf-config/CirrusSearch-common.php: https://gerrit.wikimedia.org/r/#/c/267053/ (duration: 01m 10s) [production]
00:42 <krenair@mira> Synchronized tests/cirrusTest.php: https://gerrit.wikimedia.org/r/#/c/267053/ (duration: 01m 11s) [production]
00:35 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/267025/ (duration: 01m 12s) [production]
00:25 <krenair@mira> Synchronized php-1.27.0-wmf.11/extensions/Graph/modules/graph2.js: https://gerrit.wikimedia.org/r/#/c/267065/ (duration: 01m 11s) [production]
00:17 <krenair@mira> Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/267060/ (duration: 01m 12s) [production]
00:02 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/267189/2 (duration: 01m 11s) [production]
2016-01-28 §
23:51 <mutante> caesium - stop puppet, shutdown server, remove from icinga, clean puppet cert ... [production]
23:46 <TimStarling> on ruthenium installing build dependencies and compiling uprightdiff for test [production]
23:20 <ori@mira> Synchronized php-1.27.0-wmf.11/includes/api/ApiStashEdit.php: Ia4196eba9: Add ParserOutputStashForEdit hook for extension cache warming (duration: 01m 10s) [production]
23:17 <tgr@mira> Synchronized php-1.27.0-wmf.11/includes/session/SessionManager.php: T125161 (duration: 01m 11s) [production]
22:58 <ottomata> restoring MobileWebSectionUsage_14321266 from db1047 to dbstore1002 using mysqlimport [production]
22:23 <bblack> starting cache_mobile->cache_text conversion in eqiad - https://phabricator.wikimedia.org/T109286 [production]
22:09 <bblack> eqiad pybal->etcd conversion done [production]
22:01 <dduvall@mira> Synchronized php-1.27.0-wmf.11/extensions/WikimediaEvents/WikimediaEventsHooks.php: deploying fix for T125151 (duration: 01m 15s) [production]
21:59 <mutante> releases.wm.org - switched backend to bromine [production]
21:58 <bblack> converting active eqiad LVS/pybal to etcd [production]
21:55 <mutante> caesium - stopped apache [production]
21:31 <ori@mira> Synchronized php-1.27.0-wmf.11/extensions/AbuseFilter: I13fcc3ce4: Updated mediawiki/core Project: mediawiki/extensions/AbuseFilter 19baa3b6e51b8fe6baf6e3ce7e590060e8e6eec9 (duration: 01m 11s) [production]
21:27 <bblack> converting backup/inactive eqiad LVS/pybal to etcd [production]
21:16 <dduvall@mira> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.11 [production]
20:54 <mutante> sca1001 - stop mathoid,graphoid,citoid [production]
20:52 <mutante> sca1002 - stop mathoid,graphoid,citoid [production]
20:50 <dduvall@mira> Synchronized php-1.27.0-wmf.11: syncing 1.27.0-wmf.11 for T125114 and https://gerrit.wikimedia.org/r/#/c/267128/ (duration: 03m 30s) [production]
20:25 <bblack> depool -> reboot cp4008 (ulsfo text, trying new kernel with live traffic) [production]
20:00 <bblack> depool -> reboot cp4011 (ulsfo mobile, currently unused for traffic - testing local conftool-scripts depool + new kernel) [production]
19:55 <ori@mira> Synchronized wmf-config: Iea2573ccfbe: Revert "Autopromotion: remove deprecated onView event, fix INGROUPS" (duration: 02m 13s) [production]