1051-1100 of 2555 results (12ms)
2014-07-22 §
14:02 <hashar> rebooting deployment-cache-upload02 varnish not happy with memory mapping [releng]
13:51 <hashar> rebooting bits varnish cache [releng]
13:43 <hashar> rebased puppetmaster repo. Rebase got broken after ''0317463 - beta: New script to restart apaches'' got merged in. [releng]
13:35 <hashar> apt-get upgrade on deployment-cache-bits01 + varnish upgrade [releng]
09:28 <hashar> Removing role::beta::natfix that is now handled by labs DNS and the class is removed with {{gerrit|146091}} [releng]
2014-07-21 §
23:37 <ori> Switched over beta cluster app servers to HHVM [releng]
21:27 <bd808> Killed update.php jobs; Antoine will give jobs a longer timeout [releng]
21:23 <bd808> Running update.php for simplewiki in screen [releng]
21:22 <bd808> Running update.php for hewiki in screen [releng]
21:21 <bd808> Running update.php for eswiki in screen [releng]
21:21 <bd808> Running update.php for cawiki in screen [releng]
21:21 <bd808> Running update.php for commonswiki in screen [releng]
21:18 <hashar> Restarting upd2log-mw on deployment-bastion. There is a bunch of [python] <defunct> processes [releng]
17:32 <bd808> Updated scap to 4871208 (+ cherry pick of I6a56b5e) [releng]
17:12 <bd808> Hotfix for scap ssh host key checking to fix jenkins scap job [releng]
17:03 <bd808> Testing scap change I40a891b via cherry-pick [releng]
10:25 <hashar> on bastion, fixed some puppet dependency to have nutcracker to start with the proper configuration {{gerrit|148043}} [releng]
10:20 <hashar> upgrading packages on deployment-bastion [releng]
10:19 <hashar> deleted /var/lib/apt/lists/lock on bastion. Was prevent apt-get update from running [releng]
10:18 <hashar> setting up nutcracker on deployment-bastion. It was installed but the puppet class to configure it was not being applied. Related Gerrit patches: {{gerrit|148041}} and {{gerrit|148042}} [releng]
09:25 <hashar> rebooting deployment-apache02 [releng]
09:22 <hashar> rebooting deployment-apache01. [releng]
00:27 <ori> deployment-mediawiki01 & deployment-mediawiki02: configured for project-local puppet & salt masters [releng]
2014-07-18 §
00:30 <bd808> removed local l10nupdate user from deployment-jobrunner01 and deployment-videoscaler01 [releng]
00:22 <bd808> Killed stuck beta-update-databases-eqiad job ( stuck for over 60m waiting for executor; deadlock?) [releng]
00:21 <ori> beta broke due to I433826423. app servers load prod apache confs from /etc/apache2/wikimedia. temp fix: locally hack apache2.conf to load /usr/local/apache2/conf/all.conf; disable puppet. [releng]
2014-07-17 §
23:18 <bd808> Puppet broken for deployment-bastion by labs specific logic in misc::deployment::vars. [releng]
19:01 <mwalker> possibly breaking labs by cherry picking an apparmor patch that affects mysql https://gerrit.wikimedia.org/r/#/c/147027/ [releng]
2014-07-16 §
19:15 <mwalker> updated puppet about 20 minutes ago for new ocg variables (now officially in production puppet instead of just cherry picked) [releng]
2014-07-15 §
18:26 <bd808> Removed local mwdeploy user from /etc/passwd on deployment-videoscaler01 and deployment-jobrunner01 [releng]
16:59 <bd808> scap failing to deploymnet-videoscaler01 and deploymnet-jobrunner01 due to other random failures now. Lots of strange permissions errors during rsync [releng]
16:37 <bd808> scap failing to deploymnet-videoscaler01 and deploymnet-jobrunner01 due to ssh auth failures; likely a puppet config problem [releng]
2014-07-10 §
22:37 <bd808> Added Gergő Tisza and Yuvipanda as project admins [releng]
2014-07-08 §
23:37 <bd808> Updated Kibana to 0afda49 (latest upstream head) [releng]
17:03 <greg-g> Added John F. Lewis to the project after his NDA was signed by Mark (RT 7722) [releng]
2014-07-07 §
20:55 <bd808> Killed stuck `apt-get update` job on deployment-jobrunner01 started on Jun17 [releng]
20:20 <bd808> Fixed puppet on deployment-analytics01 with manual apt-get commands. [releng]
20:08 <bd808> Ran `apt-get dist-upgrade` on deployment-analytics01 to upgrade hadoop, hive, pig, etc which were failing to update via puppet. [releng]
2014-07-04 §
02:28 <RoanKattouw> Unbroke replication on deployment-db2, it's catching up now [releng]
2014-07-03 §
18:59 <legoktm> manually created centralauth.renameuser_status table [releng]
16:04 <bd808> Updated scap to ff04431 [releng]
09:24 <hashar> Reindexed ElasticSearch index for cawiki/eswiki with: mwscript extensions/CirrusSearch/maintenance/forceSearchIndex.php --wiki {cawiki,eswiki} --batch-size=50 [releng]
09:10 <hashar> used addwiki.php to create the wiki. manually triggered the Jenkins job that update the databases https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/2319/ [releng]
09:06 <hashar> Adding cawiki and eswiki for cxserver testing {{gerrit|Ibbcbd43d43f878099788518d9e952fa7b20d8ca3}} [releng]
2014-07-02 §
07:49 <hashar> cxserver being configured! {{gerrit|140723}} by Kartik and Niklas \\O/ [releng]
2014-07-01 §
15:46 <bd808> Fixed git rebase conflict in operations/puppet on deployment-salt [releng]
13:29 <manybubbles> rebuilding Cirrus search index in beta to pick up new configuration and cache warmers [releng]
11:20 <hashar> Added Filippo Giunchedi to the project as an admin (WMF ops) [releng]
2014-06-30 §
20:47 <bd808> The state of puppet for beta is badly broken. I have hacked things to get puppet to apply on deployment-apache0[12] but puppet won't apply on deployment-bastion in part due to the same hacks. [releng]
18:48 <bd808> Created symlink /apache -> /usr/local/apache on deployment-apache0[12] to fix docroot symlinks [releng]