5151-5200 of 10000 results (57ms)
2016-04-29 ยง
19:29 <gehel> restarting elasticsearch server elastic1026.eqiad.wmnet (T110236) [production]
19:07 <gehel> restarting elasticsearch server elastic1025.eqiad.wmnet (T110236) [production]
18:21 <jzerebecki@tin> Synchronized php-1.27.0-wmf.22/extensions/Wikidata/extensions/Wikibase/repo/includes/Hooks/OutputPageBeforeHTMLHookHandler.php: wmf.22 fc20c54f7915b94ec0d15ef17e207c116910623d 2 of 2 T132645 (duration: 00m 28s) [production]
18:20 <jzerebecki@tin> Synchronized php-1.27.0-wmf.22/extensions/Wikidata/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php: wmf.22 fc20c54f7915b94ec0d15ef17e207c116910623d 1 of 2 T133924 (duration: 00m 29s) [production]
18:14 <jzerebecki@tin> Synchronized php-1.27.0-wmf.22/extensions/Wikidata/extensions/Wikibase/repo/includes/Hooks/OutputPageBeforeHTMLHookHandler.php: wmf.22 fc20c54f7915b94ec0d15ef17e207c116910623d 2 of 2 T132645 (duration: 00m 34s) [production]
18:14 <robh> started all slaves via dbstore2001 this time. [production]
18:12 <jzerebecki@tin> Synchronized php-1.27.0-wmf.22/extensions/Wikidata/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php: wmf.22 fc20c54f7915b94ec0d15ef17e207c116910623d 1 of 2 T133924 (duration: 00m 44s) [production]
18:07 <robh> started all slaves via dbstore2002 per jaime's request [production]
17:45 <gehel> restarting elasticsearch server elastic1024.eqiad.wmnet (T110236) [production]
16:56 <gehel> restarting elasticsearch server elastic1023.eqiad.wmnet (T110236) [production]
16:22 <gehel> restarting elasticsearch server elastic1022.eqiad.wmnet (T110236) [production]
15:29 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db2047 and db2068. Depool db2008, db2009. Pool db2033 as the new x1 node. (duration: 00m 27s) [production]
15:17 <gehel> restarting elasticsearch server elastic1021.eqiad.wmnet (T110236) [production]
14:56 <oblivian@palladium> conftool action : set/pooled=yes; selector: name=mw1153.eqiad.wmnet [production]
14:54 <jynus> moving topology of db2033 to be the new x1 master on codfw [production]
14:40 <oblivian@palladium> conftool action : set/pooled=no; selector: name=mw1153.eqiad.wmnet [production]
14:32 <gehel> restarting elasticsearch server elastic1020.eqiad.wmnet (T110236) [production]
14:26 <hashar> Rebased tin:/srv/mediawiki-staging 31886c7..8e2670a . Bring in 3 changes that are solely for beta cluster. [production]
13:54 <jynus> stopping mysql db2008 (cloning to db2033) [production]
13:39 <jynus> reimaging db2033 [production]
13:09 <gehel> restarting elasticsearch server elastic1019.eqiad.wmnet (T110236) [production]
12:30 <gehel> restarting elasticsearch server elastic1018.eqiad.wmnet (T110236) [production]
11:39 <elukey> soft reboot for mw1119 (not responsive to ssh, root login timed out on the console) [production]
09:43 <gehel> restarting elasticsearch server elastic1017.eqiad.wmnet (T110236) [production]
09:42 <gehel> restarting elasticsearch server elastic1016.eqiad.wmnet (T110236) [production]
09:01 <jynus> changing live configuration of db1049 thread_pool_stall_limit to 10 to test impact on connection timout [production]
08:20 <gehel> restarting elasticsearch server elastic1016.eqiad.wmnet (T110236) [production]
07:57 <elukey> puppet disabled on new kafka codfw instances due to errors while starting Event Bus (hosts not in service) [production]
07:54 <moritzm> enabled base::firewall on stat1002 [production]
07:52 <gehel> restarting elasticsearch server elastic1015.eqiad.wmnet (T110236) [production]
07:36 <godog> stop cleanups on restbase1014-b [production]
06:46 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Reduce normal traffic on s2 API servers (duration: 00m 27s) [production]
06:33 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1038, increase weight of new hardware slaves db107[4-8] (duration: 00m 33s) [production]
05:42 <gehel> restarting elasticsearch server elastic1014.eqiad.wmnet (T110236) [production]
05:41 <mutante> re: "02:29 Krenair: last deployment was slow because of snapshot1007 being offline" it's back, i don't know why, it was powered down and i just tried switching it on. that helped. the command is literally "power on" on HP [production]
05:39 <mutante> snapshot1007 - was powered down, powering it on. (..connect to mgmt.. "damn it's a HP") [production]
05:34 <mutante> snapshot1007 - not reachable, duration 10h [production]
04:58 <gehel> restarting elasticsearch server elastic1013.eqiad.wmnet (T110236) [production]
02:29 <Krenair> last deployment was slow because of snapshot1007 being offline, icinga shows it's been like that for the last 7 hours [production]
02:22 <krenair@tin> Synchronized php-1.27.0-wmf.22/extensions/EventBus: https://gerrit.wikimedia.org/r/286115 (duration: 02m 27s) [production]
02:19 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.22) (duration: 09m 20s) [production]
00:27 <mutante> RT - remove libapache2-mod-php5, restart Apache, Perl apps dont need PHP [production]
00:22 <aaron@tin> Synchronized wmf-config/filebackend-production.php: Set "autoResync" on for local-multiwrite (duration: 02m 29s) [production]
00:15 <cwd> updated civicrm from 777a91b8f9f6003a3eebdb8f2c73e45cc2bfb4a4 to b386a6821c71310950ccdcdcf2616add727e1af4 [production]
00:04 <Dereckson> Previous deployment: [[Gerrit:285553]] Enable lazy loaded references in beta (T129693) [production]
00:03 <Dereckson> Previous deployment: [[Gerrit:285927]] GoogleNewsSitemap configuration (T39608) [production]
00:03 <Dereckson> Previous deployment: [[Gerrit:252627]] Revert "Increase abusefilter emergency disable threshold on MediaWiki.org" [production]
00:03 <Dereckson> Previous deployment: [[Gerrit:280865]]+[[Gerrit:285989]] Allow wmf-config/throttle.php to be lenient on ip/IP typo, clean rules (no-op) [production]
00:02 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 02m 25s) [production]
00:02 <Dereckson> Previous deployment: [[Gerrit:279142]] Document FIXME statement in config (no-op) [production]