|
2014-03-24
§
|
| 23:10 |
<hashar> |
l10n cache got broken due to a PHP fatal error I introduced. It is back up now. Found out via https://integration.wikimedia.org/dashboard/ |
[releng] |
| 23:09 |
<hashar> |
upgraded all pmtpa varnishes, ran puppet on all of them. all set! |
[releng] |
| 22:57 |
<bd808> |
Forced /srv/scap to update to c771a46 across the cluster |
[production] |
| 22:57 |
<hashar> |
restarting deployment-cache-upload04 , apparently stalled\t |
[releng] |
| 22:48 |
<hashar> |
upgrading varnish on all pmtpa caches. |
[releng] |
| 22:47 |
<hashar> |
apt-get upgrade varnish on deployment-cache-bits03 |
[releng] |
| 22:45 |
<marktraceur> |
attempted restart of varnish on betalabs; seems to have failed, trying again |
[releng] |
| 22:42 |
<hashar> |
made marktraceur a project admin and granted sudo rights |
[releng] |
| 22:39 |
<marktraceur> |
Restarting betalabs varnish to workaround https://bugzilla.wikimedia.org/show_bug.cgi?id=63034 |
[releng] |
| 20:07 |
<subbu> |
deployed Parsoid fa03dd20 with deploy repo sha e4d28e7e |
[production] |
| 19:55 |
<mutante> |
DNS update - removing tampa search pools and searchidx2 |
[production] |
| 19:36 |
<mutante> |
shutdown searchidx2 |
[production] |
| 19:23 |
<ottomata> |
stopping hadoop service on analytics1020 and shutting down for move to Row D |
[production] |
| 19:19 |
<mutante> |
searchidx2 - revoked puppet cert,remove from puppet,icinga,salt... |
[production] |
| 19:02 |
<ottomata> |
stopping hadoop services on analytics1019 and shutting it down for move to Row D |
[production] |
| 18:22 |
<springle> |
synchronized wmf-config/db-eqiad.php 's5 db1037 full steam' |
[production] |
| 17:25 |
<bd808> |
Converted deployment-db1.eqiad.wmflabs to use local puppet & salt masters |
[releng] |
| 17:06 |
<bd808> |
Changed rules in sql security group to use CIDR 10.0.0.0/8. |
[releng] |
| 17:05 |
<bd808> |
Changed rules in search security group to use CIDR 10.0.0.0/8. |
[releng] |
| 17:05 |
<bd808> |
Built deployment-elastic04.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 16:19 |
<bd808> |
Built deployment-elastic03.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 16:08 |
<bd808> |
Built deployment-elastic02.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 15:54 |
<bd808> |
Built deployment-elastic01.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 14:31 |
<ottomata> |
stopping hadoop services and shutting down analytics1018 |
[production] |
| 10:31 |
<hashar> |
migrated deployment-solr to self puppet/salt masters |
[releng] |
| 03:03 |
<springle> |
synchronized wmf-config/db-eqiad.php 's5 repool db1037 warm up' |
[production] |
| 02:41 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Mar 24 02:41:13 UTC 2014 (duration 41m 12s) |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.23wmf19) at 2014-03-24 02:17:32+00:00 |
[production] |
|
2014-03-21
§
|
| 22:13 |
<K4-713> |
updated prod civicrm to 2b635c1f5cf7 |
[production] |
| 22:01 |
<K4-713> |
updated prod civicrm to c736116cac50121d |
[production] |
| 19:23 |
<mutante> |
torrus was broken. going through fix per https://wikitech.wikimedia.org/wiki/Torrus#Deadlock_problem |
[production] |
| 18:14 |
<mutante> |
DNS update - adding zero.wikimedia.org |
[production] |
| 18:05 |
<mutante> |
disabled and commented mobile2-5 in pmtpa pybal |
[production] |
| 18:00 |
<Krinkle> |
Reloading Zuul to deploy I20b4aa9159df7 |
[production] |
| 17:54 |
<mutante> |
same for search_pool5, and setting search_prefix (search19/20) to disabled |
[production] |
| 17:52 |
<mutante> |
removing search_pool4 from pmtpa pybal |
[production] |
| 17:51 |
<mutante> |
removing owa1-3 from pmtpa pybal |
[production] |
| 17:42 |
<mutante> |
removing pmtpa https from pybal (ssl1-4) |
[production] |
| 17:23 |
<mutante> |
remove pmtpa bits (sq67-70) from pybal |
[production] |
| 17:22 |
<Krinkle> |
Running deleteEqualMessages.php on wuuwiki (bug 43917 comment 23) |
[production] |
| 14:08 |
<hashar> |
Jenkins: label labs slaves with hasJenkinsDebianGlue to build debian packages on them |
[production] |
| 13:43 |
<hashar> |
Jenkins: installing jenkins-debian-glue and misc::package-builder on labs slaves. |
[production] |