|
2016-11-15
§
|
| 07:47 |
<marostegui> |
Deploy schema change s4 commonswiki.revision db1064 - T147305 |
[production] |
| 06:45 |
<yuvipanda> |
restarted with webservice stop && webservice --backend=kubernetes start |
[tools.pub] |
| 06:13 |
<Krinkle> |
Creating and provisioning cvn-app7 (m1.large; ubuntu-14.04-trusty ) - replacement for cvn-app5 (m1.xlarge; ubuntu-12.04-precise) |
[cvn] |
| 06:00 |
<Krinkle> |
Decom cvn-app4 (created 2014; ubuntu-12.04-precise) - all bots migrated to cvn-app6 (ubuntu-14.04-trusty) |
[cvn] |
| 06:00 |
<Krinkle> |
Moving CVNBot23 from cvn-app4 to cvn-app6 |
[cvn] |
| 05:55 |
<Krinkle> |
Moving CVNBot9, CVNBot10, and CVNBot19 from cvn-app4 to cvn-app6 |
[cvn] |
| 05:40 |
<Krinkle> |
Moving CVNBot6, CVNBot7, and CVNBot8 from cvn-app4 to cvn-app6 |
[cvn] |
| 05:20 |
<yuvipanda> |
restart all k8s webservices too |
[tools] |
| 05:17 |
<Krinkle> |
Moving CVNBot3 from cvn-app4 to cvn-app6 |
[cvn] |
| 05:05 |
<yuvipanda> |
restarting all webservices on gridengine |
[tools] |
| 04:31 |
<bd808> |
Restarted webservice to get access to r/w NFS |
[tools.nagf] |
| 04:24 |
<mutante> |
!log (test logging) |
[production] |
| 04:24 |
<gerrit> |
316497 was merged and logging from puppetmaster1001 over to icinga still works |
[production] |
| 03:21 |
<chasemp> |
reboot tools-checker-01 |
[tools] |
| 02:56 |
<chasemp> |
reboot tools-exec-1405 to ensure noauto works (because atboot=>false is lies) |
[tools] |
| 02:31 |
<chasemp> |
reboot tools-exec-1406 |
[tools] |
| 02:18 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.2) (duration: 06m 13s) |
[production] |
| 01:12 |
<mutante> |
deploying ghostscript regression update on Videoscalers (and manually on osmium) |
[production] |
| 01:08 |
<mutante> |
deploying ghostscript regression update on Imagescalers |
[production] |
| 01:05 |
<mutante> |
deploying ghostscript regression update on API appservers |
[production] |
| 01:01 |
<mutante> |
deploying ghostscript regression update on jobrunner-eqiad |
[production] |
| 00:56 |
<mutante> |
bromine - deleted some un-packed mediawiki release versions from home/csteipp/releasetools for disk space (back tp 80%) |
[production] |
| 00:51 |
<mutante> |
bromine - apt-get clean for disk space |
[production] |
| 00:24 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/321118/ (duration: 00m 47s) |
[production] |
|
2016-11-14
§
|
| 23:30 |
<MaxSem> |
Assigned email for my bot, RoboMaxCyberSem |
[production] |
| 23:28 |
<cwd> |
updated smashpig payment listeners from 142e60bfe09982f9f86b7839cbf35d9e0e96b8d7 to a0ae95edd001d51d0cff5c080d0372c14efcf9a9 |
[production] |
| 23:18 |
<godog> |
add 150G to labsdb1004:/srv/labsdb to get it out of warning threshold T150553 |
[production] |
| 23:07 |
<godog> |
silence disk space alerts on labsdb1004 for 4h while investigating reoccurence - T150553 |
[production] |
| 22:51 |
<chasemp> |
shut down bastion 02 and 05 and make 03 root only |
[tools] |
| 21:26 |
<demon@tin> |
Synchronized scap/plugins/patch.py: Patching tool for fun and profit (duration: 00m 49s) |
[production] |
| 21:17 |
<demon@tin> |
Synchronized wmf-config/: Coding style fixes (duration: 00m 49s) |
[production] |
| 21:07 |
<mutante> |
deploying regression update for ghostscript (DSA-3691-2) on all eqiad mw appservers |
[production] |
| 21:05 |
<demon@tin> |
Synchronized fonts/: For completeness. Also for co-master git sync (duration: 01m 09s) |
[production] |
| 20:59 |
<mutante> |
deploying regression update for ghostscript (DSA-3691-2) on all codfw mw appservers |
[production] |
| 20:47 |
<mutante> |
deploying regression update for ghostscript (DSA-3691-2) on MW API appservers |
[production] |
| 20:21 |
<demon@tin> |
Synchronized .gitignore: For completeness (duration: 00m 46s) |
[production] |
| 20:15 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:321162|Allow 2FA for the abusefilter group if enabled on wiki]] (duration: 00m 53s) |
[production] |
| 19:54 |
<thcipriani> |
ran: mwscript extensions/Translate/scripts/ttmserver-export.php --wiki=frwiktionary |
[production] |
| 19:53 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320352|Enable translation memory of Translate for frwiktionary (T150146)]] (duration: 00m 47s) |
[production] |
| 19:40 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:321093|Ban ten most popular passwords from fawiki (T150570)]] (duration: 00m 46s) |
[production] |
| 19:35 |
<madhuvishy> |
Stopped cron on tools-cron-01 (T146154) |
[tools] |
| 19:17 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320993|MF Beta: Enable moving first paragraph before infobox (T149830)]] (duration: 00m 52s) |
[production] |
| 19:01 |
<demon@tin> |
Synchronized w/static.php: Removing useless function param (duration: 01m 00s) |
[production] |
| 18:24 |
<madhuvishy> |
Tools NFS is read-only. /data/project and /home across tools are ro T146154 |
[tools] |
| 16:57 |
<yuvipanda> |
stopped gridengine master |
[tools] |
| 16:47 |
<yuvipanda> |
start restarting kubernetes webservice pods |
[tools] |
| 16:30 |
<madhuvishy> |
Unmounted all nfs shares from tools-k8s-master-01 (sudo /usr/local/sbin/nfs-mount-manager clean) T146154 |
[tools] |
| 16:22 |
<yuvipanda> |
kill maintain-kubeusers on tools-k8s-master-01, sole process touching NFS |
[tools] |
| 16:22 |
<chasemp> |
enable puppet and run on tools-services-01 |
[tools] |
| 16:21 |
<yuvipanda> |
restarting all webservice jobs, watching webservicewatcher logs on tools-services-02 |
[tools] |