|
2015-04-29
§
|
| 21:03 |
<andrewbogott> |
suspending and shrinking disks of many instances |
[releng] |
| 17:15 |
<thcipriani> |
removed l10nupdate user from /etc/passwd on deployment-bastion |
[releng] |
| 15:00 |
<hashar> |
Instances are being moved out from labvirt1005 which has some faulty memory. List of instances at https://phabricator.wikimedia.org/T97521#1245217 |
[releng] |
| 14:25 |
<hashar> |
upgrading zuul on integration-slave-precise-1011 for https://phabricator.wikimedia.org/T97106 |
[releng] |
| 14:11 |
<hashar> |
rebooting integration-saltmaster stalled. |
[releng] |
| 13:11 |
<hashar> |
Rebooting deployment-parsoid05 via wikitech interface. |
[releng] |
| 13:02 |
<hashar> |
labvirt1005 seems to have hardware issue. Impacts a bunch of beta cluster / integration instances as listed on https://phabricator.wikimedia.org/T97521#1245217 |
[releng] |
| 12:22 |
<hashar> |
deployment-parsoid05 slow down is https://phabricator.wikimedia.org/T97421 . Running apt-get upgrade and rebooting it but its slowness issue might be with the underlying hardware |
[releng] |
| 12:13 |
<hashar> |
killing puppet on deployment-parsoid05 eats all CPU for some reason |
[releng] |
| 02:40 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/207363 and https://gerrit.wikimedia.org/r/207368 |
[releng] |
|
2015-04-28
§
|
| 23:33 |
<hoo> |
Ran foreachwiki extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --load-from 'http://meta.wikimedia.beta.wmflabs.org/w/api.php' to fix all sites tables |
[releng] |
| 23:18 |
<hoo> |
Ran mysql> INSERT INTO sites (SELECT * FROM wikidatawiki.sites); on enwikinews to populate the sites table |
[releng] |
| 23:18 |
<hoo> |
Ran mysql> INSERT INTO sites (SELECT * FROM wikidatawiki.sites); on testwiki to populate the sites table |
[releng] |
| 20:57 |
<YuviPanda> |
KILL KILL KILL DEPLOYMENT-LUCID-SALT WITH FIRE AND BRIMSTONE AND BAD THINGS |
[releng] |
| 17:48 |
<James_F> |
Restarting grrrit-wm for config change. |
[releng] |
| 16:24 |
<bd808> |
Updated scap to ef15380 (Make scap localization cache build $TMPDIR aware) |
[releng] |
| 15:42 |
<bd808> |
Freed 5G on deployment-bastion by deleting abandoned /tmp/scap_l10n_* directories |
[releng] |
| 14:01 |
<marxarelli> |
reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/206967/ |
[releng] |
| 00:17 |
<greg-g> |
after the 3rd or so time doing it (while on the Golden Gate Bridge, btw) it worked |
[releng] |
| 00:11 |
<greg-g> |
still nothing... |
[releng] |
| 00:10 |
<greg-g> |
after disconnecting, marking temp offline, bringing back online, and launching slave agent: "Slave successfully connected and online" |
[releng] |
| 00:07 |
<greg-g> |
deployment-bastion is idle, yet we have 3 pending jobs waiting for an executer on it - will disconnect/reconnect it in Jenkins |
[releng] |
|
2015-04-27
§
|
| 21:45 |
<bd808> |
Manually triggered beta-mediawiki-config-update-eqiad for zuul build df1e789c726ad4aae60d7676e8a4fc8a2f6841fb |
[releng] |
| 21:20 |
<bd808> |
beta-scap-equad job green again after adding a /srv/ disk to deployment-jobrunner01 |
[releng] |
| 21:08 |
<bd808> |
Applied role::labs::lvm::srv on deployment-jobrunner01 and forced puppet run |
[releng] |
| 21:08 |
<bd808> |
Deleted deployment-jobrunner01:/srv/* in preparation for applying role::labs::lvm::srv |
[releng] |
| 21:06 |
<bd808> |
deployment-jobrunner01 missing role::labs::lvm::srv |
[releng] |
| 21:00 |
<bd808> |
Root partition full on deployment-jobrunner01 |
[releng] |
| 20:53 |
<bd808> |
removed mwdeploy user from deployment-bastion:/etc/passwd |
[releng] |
| 20:15 |
<Krinkle> |
Relaunched Gearman connection |
[releng] |
| 19:53 |
<Krinkle> |
Jenkins unable to re-create Gearman connection. (HTTP 503 error from /configure). Have to force restart Jenkins |
[releng] |
| 17:32 |
<Krinkle> |
Relauch slave agent on deployment-bastion |
[releng] |
| 17:31 |
<Krinkle> |
Jenkins slave deployment-bastion deadlock waiting for executors |
[releng] |
| 08:01 |
<_joe_> |
installed hhvm 3.6 on deployment-mediawiki02 |
[releng] |
|
2015-04-24
§
|
| 18:10 |
<Krinkle> |
cvn Promited Rxy from member to projectadmin |
[releng] |
| 18:01 |
<thcipriani> |
ran sudo chown -R mwdeploy:mwdeploy /srv/mediawiki on deployment-bastion to fix beta-scap-eqiad, hopefully |
[releng] |
| 17:26 |
<thcipriani> |
remove deployment-prep from domain in /etc/puppet/puppet.conf on deployment-stream, puppet now OK |
[releng] |
| 17:20 |
<thcipriani> |
rm stale lock on deployment-rsync01, puppet fine |
[releng] |
| 17:10 |
<thcipriani> |
gzip /var/log/account/pacct.0 on deployment-bastion: ought to revisit logrotate on that instance. |
[releng] |
| 17:00 |
<thcipriani> |
rm stale /var/lib/puppet/state/agent_catalog_run.lock on deployment-kafka02 |
[releng] |
| 14:25 |
<_joe_> |
installing hhvm 3.6.1 on mediawiki-deployment01 |
[releng] |