|
2016-02-16
§
|
| 11:30 |
<jzerebecki> |
T117710 integration-saltmaster:~# salt -v '*slave-trusty*' cmd.run 'rm -rf /mnt/jenkins-workspace/workspace/mwext-testextension-hhvm-composer/src/skins/BlueSky' |
[releng] |
| 11:25 |
<hoo@tin> |
Started scap: Deploy Capiunto (master) to testwiki, test2wiki and testwikidata - T126399 |
[production] |
| 11:22 |
<joal> |
Launch last_access_uniques monthly job |
[analytics] |
| 11:13 |
<joal> |
Launch last_access_uniques daily job |
[analytics] |
| 11:08 |
<godog> |
repool restbase on restbase1007 |
[production] |
| 11:04 |
<joal> |
Deploying refinery on hdfs from stat1002 |
[analytics] |
| 11:01 |
<moritzm> |
installing nettle security updates |
[production] |
| 11:00 |
<joal> |
Deploying Refinery from tin (trying) |
[analytics] |
| 10:50 |
<godog> |
start swiftrepl commons thumbs for top50 popular size T125791 |
[production] |
| 10:14 |
<akosiaris> |
disable puppet, stop salt-minion on sca100{1,2} |
[production] |
| 09:52 |
<hashar> |
will cut the wmf branches this afternoon starting around 14:00 CET |
[releng] |
| 09:52 |
<hashar> |
will cut the wmf branches this afternoon starting around 14:00 CET |
[production] |
| 09:31 |
<hoo@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 58s) |
[production] |
| 02:34 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Feb 16 02:34:30 UTC 2016 (duration 8m 34s) |
[production] |
| 02:25 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.13) (duration: 12m 28s) |
[production] |
|
2016-02-15
§
|
| 22:54 |
<andrewbogott> |
Andrew encourages future puppet hackers to commit their local changes so that this mess doesn’t reappear |
[phabricator] |
| 22:52 |
<andrewbogott> |
stashed local changes in /var/lib/git/operations/puppet on deploy instance, checked out ‘production’ branch, updated |
[phabricator] |
| 22:32 |
<andrewbogott> |
rebooting instance phab-02 |
[phabricator] |
| 22:29 |
<andrewbogott> |
rebooting instance phab-01 |
[phabricator] |
| 21:26 |
<krinkle@tin> |
Synchronized w/static.php: (no message) (duration: 00m 58s) |
[production] |
| 20:55 |
<twentyafterfour> |
moved contents of /srv/phab/dumps into /srv/dumps |
[production] |
| 20:48 |
<twentyafterfour> |
backing up /srv/phab to /srv/phab.bak in case it needs to be restored in a hurry. Note: the backup excludes /srv/phab/repos which is backed up separately |
[production] |
| 20:47 |
<twentyafterfour> |
brought iridium up to date with a current puppet run and checked that all repositories are at the correct tag |
[production] |
| 20:27 |
<bd808> |
Updated to 3314de8; Guard against %D\d matching Pabricator echo; De-dup Phabricator echos |
[tools.stashbot] |
| 18:23 |
<volans> |
Upgrading db1022 (MariaDB and kernel) already depooled and put in scheduled downtime |
[production] |
| 16:46 |
<yurik> |
Deployed patch for T126897 |
[production] |
| 16:38 |
<volans@tin> |
Synchronized wmf-config/db-eqiad.php: depool db1022 (duration: 00m 58s) |
[production] |
| 16:34 |
<bd808> |
Ran sync-common on mw1119 after stuck scap update was killed there |
[production] |
| 16:29 |
<bd808> |
Killed stuck rsync by mw1119 that was keeping l10nudpate running |
[production] |
| 16:28 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.13) (duration: 855m 18s) |
[production] |
| 16:28 |
<jzerebecki> |
reloading zuul for 2d16ad3..3bb0afa |
[releng] |
| 16:27 |
<moritzm> |
installing postgres security updates on maps* |
[production] |
| 16:26 |
<halfak> |
restarted workers on ores-worker -02 and -03 |
[ores] |
| 16:10 |
<hashar> |
Image ci-jessie-wikimedia-1455552377 in wmflabs-eqiad is ready |
[releng] |
| 15:32 |
<_joe_> |
restarting the salt minions on all deployment targets |
[production] |
| 15:25 |
<jzerebecki> |
reloading zuul for e174335..2d16ad3 |
[releng] |
| 15:23 |
<hashar> |
Image ci-jessie-wikimedia-1455549539 in wmflabs-eqiad is ready |
[releng] |
| 15:22 |
<apergos> |
disabled puppet again on iridium, had to restore sshd config from filebucket after puppet run |
[production] |
| 15:19 |
<hashar> |
Regenerating Nodepool snapshot. Slave scripts have 0 bytes... |
[releng] |
| 15:18 |
<hoo|busy> |
Changed email for global account "Frau pomerenke" |
[production] |
| 15:04 |
<hashar> |
Slave scripts added to Nodepool instances! Image ci-jessie-wikimedia-1455548346 in wmflabs-eqiad is ready |
[releng] |
| 14:51 |
<apergos> |
phab role commented out on iridium and puppet re-enabled, no phab changes will be applied there |
[production] |
| 14:24 |
<godog> |
start restbase1007-b cassandra instance, bootstrapping T119935 |
[production] |
| 13:22 |
<joal> |
Failed deploying refinery: deploy from tin failed, need ottomata help |
[analytics] |
| 13:04 |
<joal> |
Deploying refinery |
[analytics] |
| 12:30 |
<_joe_> |
deployment master is now tin |
[production] |
| 11:57 |
<_joe_> |
switching the deployment host back to tin |
[production] |
| 11:48 |
<_joe_> |
restarted hhvm on mw1130, memory exhausted |
[production] |
| 11:28 |
<jynus> |
offlining disk 32:5 on db1063 |
[production] |
| 11:05 |
<hashar> |
Image ci-jessie-wikimedia-1455534001 in wmflabs-eqiad is ready |
[releng] |