|
2014-03-27
§
|
| 15:23 |
<hashar> |
role::beta::natfix cant run on deployment-bastion.eqiad because the ferm rules conflicts with the Augeas rules coming from udp2log :-( |
[releng] |
| 15:21 |
<hashar> |
applying role::beta::natfix on deployment-bastion.eqiad |
[releng] |
| 14:58 |
<hashar> |
fixed up role::beta::natfix . Ferm is now being applied again on various application server instances {{gerrit|121378}} |
[releng] |
| 13:58 |
<hashar> |
rebased puppetmaster git repository, reapplied ottomata live hacks. |
[releng] |
| 12:55 |
<hashar> |
mediawiki l10n cache being rebuild!!! |
[releng] |
| 12:54 |
<hashar> |
Fixed permissions on eqiad bastion for /srv/scap . Others (such as mwdeploy) could not read / execute scap scripts |
[releng] |
| 11:29 |
<hashar> |
MediaWiki code and configuration are now self updating on EQIAD cluster via Jenkins jobs. First run: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/4/console |
[releng] |
| 11:11 |
<hashar> |
deleting job beta-code-update , replaced by datacenter variants beta-code-update-pmtpa and beta-code-update-eqiad |
[releng] |
| 10:54 |
<hashar> |
Deleting job beta-update-databases , replaced by datacenter variants beta-update-databases-pmtpa and beta-update-databases-eqiad |
[releng] |
|
2014-03-26
§
|
| 19:05 |
<bd808> |
Added ottomata as a project member and admin |
[releng] |
| 15:46 |
<springle> |
deployment-db1 data loaded |
[releng] |
| 14:45 |
<bd808> |
created proxy https://logstash-beta.wmflabs.org for logstash instance |
[releng] |
| 14:17 |
<hashar> |
fixed up redis configuration in eqiad. Jobrunner is happy now: aawiki-504cd7d2: 0.9649 21.5M Creating a new RedisConnectionPool instance with id 627014dc7020485d721532dde4142d5190ba3cc1. {{gerrit|121060}} |
[releng] |
| 14:05 |
<hashar> |
udp2log functional on eqiad beta cluster \\O/ |
[releng] |
| 13:55 |
<hashar> |
stopping udp2log on eqiad bastion, starting udp2log-mw (really should fix that issue one day) |
[releng] |
| 13:52 |
<hashar> |
dropped some live hack on eqiad in /data/project/apache/common-local and ran git pull |
[releng] |
| 13:14 |
<hashar> |
Dropping enwikivoyage and dewikivoyage databases from sql02. Related changes are updating the Jenkins config: https://gerrit.wikimedia.org/r/#/c/121045/ and cleaning up the mw-config : https://gerrit.wikimedia.org/r/#/c/121047/ |
[releng] |
| 07:53 |
<springle> |
installed mariadb via puppet on deployment-db1. no data yet |
[releng] |
|
2014-03-24
§
|
| 23:16 |
<marktraceur> |
Touching all the MMV scripts because they're not getting invalidated or something |
[releng] |
| 23:10 |
<hashar> |
l10n cache got broken due to a PHP fatal error I introduced. It is back up now. Found out via https://integration.wikimedia.org/dashboard/ |
[releng] |
| 23:09 |
<hashar> |
upgraded all pmtpa varnishes, ran puppet on all of them. all set! |
[releng] |
| 22:57 |
<hashar> |
restarting deployment-cache-upload04 , apparently stalled\t |
[releng] |
| 22:48 |
<hashar> |
upgrading varnish on all pmtpa caches. |
[releng] |
| 22:47 |
<hashar> |
apt-get upgrade varnish on deployment-cache-bits03 |
[releng] |
| 22:45 |
<marktraceur> |
attempted restart of varnish on betalabs; seems to have failed, trying again |
[releng] |
| 22:42 |
<hashar> |
made marktraceur a project admin and granted sudo rights |
[releng] |
| 22:39 |
<marktraceur> |
Restarting betalabs varnish to workaround https://bugzilla.wikimedia.org/show_bug.cgi?id=63034 |
[releng] |
| 17:25 |
<bd808> |
Converted deployment-db1.eqiad.wmflabs to use local puppet & salt masters |
[releng] |
| 17:06 |
<bd808> |
Changed rules in sql security group to use CIDR 10.0.0.0/8. |
[releng] |
| 17:05 |
<bd808> |
Changed rules in search security group to use CIDR 10.0.0.0/8. |
[releng] |
| 17:05 |
<bd808> |
Built deployment-elastic04.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 16:19 |
<bd808> |
Built deployment-elastic03.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 16:08 |
<bd808> |
Built deployment-elastic02.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 15:54 |
<bd808> |
Built deployment-elastic01.eqiad.wmflabs with local salt/puppet master, secondary disk on /var/lib/elasticsearch and role::elasticsearch::server |
[releng] |
| 10:31 |
<hashar> |
migrated deployment-solr to self puppet/salt masters |
[releng] |
|
2014-03-20
§
|
| 23:46 |
<bd808> |
Mounted secondary disk as /var/lib/elasticsearch on deployment-logstash1 |
[releng] |
| 23:46 |
<bd808> |
Converted deployment-tin to use local puppet & salt masters |
[releng] |
| 22:09 |
<hashar> |
Migrated videoscaler01 to use self salt/puppet masters. |
[releng] |
| 21:30 |
<hashar> |
manually installing timidity-daemon on jobrunner01.eqiad so puppet can stop it and stop whining |
[releng] |
| 21:00 |
<hashar> |
migrate jobrunner01.eqiad.wmflabs to self puppet/salt masters |
[releng] |
| 20:55 |
<hashar> |
deleting deployment-jobrunner02 , lets start with a single instance for nwo |
[releng] |
| 20:51 |
<hashar> |
Creating deployment-jobrunner01 and 02 in eqiad. |
[releng] |
| 15:47 |
<hashar> |
fixed salt-minion service on deployment-cache-upload01 and deployment-cache-mobile03 by deleting /etc/salt/pki/minion/minion_master.pub |
[releng] |
| 15:30 |
<hashar> |
migrated deployment-cache-upload01.eqiad.wmflabs and deployment-cache-mobile03.eqiad.wmflabs to use the salt/puppetmaster deployment-salt.eqiad.wmflabs. |
[releng] |
| 15:30 |
<hashar> |
deployment-cache-upload01.eqiad.wmflabs and deployment-cache-mobile03.eqiad.wmflabs recovered!! /dev/vdb does not exist on eqiad which caused the instance to be stalled. |
[releng] |