9301-9350 of 10000 results (45ms)
2013-11-04 §
20:34 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.23wmf2 [production]
20:29 <reedy> synchronized wmf-config/InitialiseSettings.php 'Enable BetaFeatures and friends on mediawikiwiki and testwikidatawiki' [production]
20:22 <Reedy> Created BetaFeatures tables on mediawikiwiki and testwikidatawiki [production]
20:19 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: mediawikiwiki and testwikidatawiki to 1.23wmf2 [production]
19:58 <mutante> restarted gitblit on antimony [production]
19:48 <gwicke> updated Parsoid to d7b556f25353 [production]
18:47 <awight> updated crm from 272966e353244e668a64c54ca4fa6f344079f4bf to 8d9240fea273faf434881a16d894ef0ad6c616c0 [production]
18:30 <mark> Configured cr1-esams and cr2-knams for new LVS service IP range 91.198.174.192/27 [production]
18:25 <aaron> synchronized php-1.23wmf2/maintenance/cleanupUploadStash.php 'f3332a3d932c6919b3311d89edee20f886e4e86e' [production]
17:58 <awight> rollback crm to 272966e353244e668a64c54ca4fa6f344079f4bf [production]
17:57 <awight> updated crm from 272966e353244e668a64c54ca4fa6f344079f4bf to 63de2d6b8bc92d6c2130a242ff4995ba71b5ce11 [production]
17:49 <mark> Deactivated ae1.101 on cr1-esams and cr2-knams [production]
16:42 <cmjohnson1> dns update [production]
16:31 <mark> Disabled multicast routing between eqiad and pmtpa, setup udpmcast between chromium and dobson instead [production]
14:44 <mark> Disabled multicast traffic reduction on csw1-sdtpa [production]
14:41 <akosiaris> removed LACP configuration from asw-b-eqiad for palladium and set it to standard access-port and private1-b-eqiad vlan [production]
14:41 <akosiaris> removed LACP configuration from asw-a-eqiad for strontium and set it to standard access-port and private1-a-eqiad vlan [production]
14:24 <mark> Killed udpmcast on chromium [production]
14:13 <mark> Configured OSPF/OSPF3 on cr1-eqiad:xe-4/2/2 <--> cr2-knams:xe-1/1/0 [production]
13:01 <hashar> Jenkins upgrading gearman (0.0.4 -> 0.0.5) plugin on gallium from http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/gearman-plugin/0.0.5/ [production]
11:22 <mark> multicast routing to pmtpa restored for now [production]
10:52 <mark> Deactivated BFD on flapping link ae0 between cr1-eqiad and cr2-eqiad [production]
10:47 <mark> Deactivated anycast PIM RP on cr2-eqiad [production]
10:21 <mark> Reenabled 10G wave between cr2-eqiad and cr1-sdtpa, set high OSPF metrics instead [production]
09:56 <akosiaris> added python-apscheduler to apt.wikimedia.org [production]
09:48 <hashar> Jenkins: upgrading PHP_CodeSniffer from 1.4.6 to 1.4.7 [production]
08:32 <reedy> synchronized php-1.23wmf2/ 'Ibcf77ed7f04c14a477d7cfd0e244929c552c3394' [production]
08:00 <paravoid> cr2-eqiad set xe-5/2/1 disable; 10g wave to cr1-sdtpa, flapping since yesterday, causing packet loss and outages [production]
06:24 <jeremyb> left morebots running on tools-login because it doesn't work on all grid hosts. see https://gerrit.wikimedia.org/r/93426 [production]
05:58 <jeremyb> test2 [production]
05:50 <jeremyb> test [production]
05:08 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Nov 4 05:08:07 UTC 2013 [production]
04:40 <LocalisationUpdate> completed (1.23wmf2) at Mon Nov 4 04:40:40 UTC 2013 [production]
04:30 <LocalisationUpdate> completed (1.23wmf1) at Mon Nov 4 04:30:48 UTC 2013 [production]
04:19 <ori-l> l10nupdate failures are due to "No submodule mapping found in .gitmodules for path 'WikibaseDatabase'"; repo appears to have been deleted; fixed with git rm --cached WikibaseDatabase as l10nupdate on tin. [production]
04:15 <LocalisationUpdate> failed: git pull of extensions failed [production]
03:12 <Tim> moved udpmcast to chromium since it actually has an external IP address [production]
03:05 <Tim> moved udpmcast.py from dobson to tungsten to work around failure of multicast routing eqiad -> pmtpa [production]
02:01 <LocalisationUpdate> failed: git pull of extensions failed [production]
2013-11-03 §
12:50 <mark> Activated BFD between cr2-eqiad and cr1-sdtpa link [production]
10:07 <mark> Send traffic from Japan and Korea to ulsfo, https://gerrit.wikimedia.org/r/#/c/93309/ [production]
07:26 <ori-l> kaulen's apache was down (killed by oom killer) & refused to start. copied over old config per #5011, restarted apache, & disabled puppet for now. [production]
07:07 <p858snake|l> Bugzilla isn't loading [production]
02:02 <LocalisationUpdate> failed: git pull of extensions failed [production]
2013-11-02 §
19:50 <cmjohnson1> dns update [production]
16:14 <reedy> synchronized wmf-config/CommonSettings.php [production]
12:43 <springle> start xtrabackup db52->db69 and db39->db71 [production]
09:56 <apergos> restarted twemproxy on ~10 of the api servers in eqiad in the lower memory/core group, was seeing a lot of entries in memcached-serious log and some processes churning on that [production]
07:37 <apergos> restarted a bunch of the apaches on the 12gb memory apis that were maxed out at their 100 client limit [production]
07:09 <apergos> and increased again to 30, after looking at memory on both groups of boxes. [production]