1451-1500 of 10000 results (39ms)
2016-03-04 §
21:01 <mutante> bast2001 - rebooting into PXE for T128899 [production]
20:00 <volans> Added logging to post-merge hook on palladium T128895 [production]
17:28 <ebernhardson@tin> Synchronized wmf-config/CirrusSearch-labs.php: prod nop, enables https in beta cluster for elasticsearch connections (duration: 00m 33s) [production]
17:25 <jynus> chgrp recursive on tin to wikidev on .git/objects [production]
16:25 <jynus> changing in a hot way db1047 replication filters [production]
15:58 <ottomata> puppet disabled on stat1003 for reportupdater deployment, paused until dan is out of meetings [production]
14:56 <cmjohnson1> rebooting iron to fix virtual console problem [production]
14:34 <jynus> upgrade and restart dbstore2002 to apply new replication filters [production]
14:28 <apergos> all services back in operation from dataset1001 [production]
14:20 <apergos> web service restored for dumps/download.wikimedia.org [production]
14:16 <moritzm> installing perl security updates [production]
14:04 <moritzm> installing postgres security updates on labsdb1004 [production]
14:02 <bblack> puppet back online for all caches (ipsec changes complete) [production]
13:48 <moritzm> installing pillow security updates [production]
13:41 <bblack> disabling puppet on esams,ulsfo,codfw caches for ipsec changes, to minimize alertspam... [production]
13:39 <urandom> canceling doomed bootstrap on restbase1009-a.eqiad.wmnet [production]
13:31 <apergos> dumps/download wikimedia.org service interrupted now while server is being upgraded [production]
13:03 <apergos> nfs filesystem from dataset1001 now unavailable as we prep for upgrade [production]
11:19 <jynus> deploying new replication check algorithm cross-fleet [production]
10:30 <volans> Start copying data from es200[124] to es201[123] (ETA ~16-17h) T127330 [production]
10:07 <volans@tin> Synchronized wmf-config/db-codfw.php: Update codfw external storage servers topology T127330 (duration: 00m 39s) [production]
09:10 <moritzm> re-imaging iron with jessie [production]
08:41 <jynus> downtiming all mysql replicas lag for 2 hours to test new alert check [production]
06:24 <mutante> gerrit being restarted for config change 274741 [production]
03:34 <urandom> Starting `nodetool cleanup' on restbase100{1,2,7-a,7-b}.eqiad.wmnet and restbase1010-a : T95253 [production]
03:28 <urandom> starting decomission of restbase1009.eqiad.wmnet : T95253 [production]
02:33 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Mar 4 02:33:49 UTC 2016 (duration 7m 44s) [production]
02:26 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.15) (duration: 11m 22s) [production]
00:35 <awight@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: followup Namespace configuration on wuu.wikipedia (duration: 00m 26s) [production]
00:34 <RoanKattouw> Running extensions/Echo/maintenance/backfillUnreadWikis.php on all wikis. This will probably take a few days [production]
00:23 <awight@tin> Synchronized wmf-config: SWAT: Disable useless Echo eventlogging schema; Site name configuration on wuu.wikipedia; Set default completion suggester scoring for beta and prod (take 2) (duration: 00m 32s) [production]
00:12 <awight@tin> Synchronized wmf-config: SWAT: Disable useless Echo eventlogging schema; Site name configuration on wuu.wikipedia; Set default completion suggester scoring for beta and prod (duration: 00m 36s) [production]
2016-03-03 §
22:36 <twentyafterfour> applying hotfix for T128751 (link ../../../extensions/Gerrit* to /srv/phab/phabricator/src/extensions/Gerrit*) - will submit puppet patch to make this permanent. [production]
22:01 <urandom> rolling restart of restbase production complete : T127387 [production]
21:53 <urandom> rolling restart of restbase production (config change) : T127387 [production]
21:45 <urandom> rolling restart of restbase staging complete : T127387 [production]
21:40 <urandom> rolling restart of restbase staging (config change) : T127387 [production]
21:38 <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Enable VisualEditor Single Edit Tab on officewiki https://gerrit.wikimedia.org/r/#/c/274814/ (duration: 00m 33s) [production]
20:26 <urandom> rolling restart of restbase production complete : T127387 [production]
20:22 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: (no message) [production]
20:22 <demon@tin> Synchronized php-1.27.0-wmf.15/extensions/Echo: Roan made me do it (duration: 00m 53s) [production]
20:20 <urandom> rolling restart of restbase production to apply config change : T127387 [production]
20:03 <urandom> rolling restart of restbase staging complete : T127387 [production]
19:58 <urandom> rolling restart of restbase in staging to apply config change : T127387 [production]
19:44 <gehel> elastic1031.eqiad.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) [production]
19:27 <volans> Completed migration of data from es200[68] to es201[579], added es201[579] to tendril. T127330 [production]
19:08 <csteipp@tin> Synchronized wmf-config/CommonSettings-labs.php: (no message) (duration: 00m 40s) [production]
19:05 <chasemp> create temp 3T lv on labstore2001 to store test backup deltas [production]
18:41 <gehel> elastic1030.eqiad.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) [production]
18:14 <urandom> lowering outbound stream throughput limit on restbase1010-a.eqiad.wmnet to 25mbps : T128107 T95253 [production]