Tatsuhiko Miyagawa's blog

Month

June 2013

4 posts

Carton and perl version support: miyagawa/heroku-buildpack-perl → github.com

Dependencies can be declared using cpanfile and frozen with carton install. The buildpack will install these dependencies using Carton into a cached ./local directory.

I implemented the experimental buildpack update to enable Carton and perl versions support.

This enables the use of Carton (dev release on v0.9) to deploy dependencies recorded in carton.lock (you have to check it in to the git repo), and then use the file .perl-version ala plenv to specify the perl runtime version. Only the latest minor point release of all stable releases are supported. See the README for details.

This has been one of my TODOs to make this buildpack actually usable on Heroku, since otherwise a) installation will use the latest cpan modules from CPAN unless you explicitly specify version in cpanfile (which is great) and b) the perl used to run your application is always /usr/bin/perl version 5.10 that comes with whatever OS that runs on Heroku.

I tried to use vulcan to build binaries on Heroku, but it looks a little complicated and requires setting up Heroku addons. I went with the simplest technique to run perl-build on heroku run bash like explained in this article, then scp back the resulting binary to my host.

This makes the perl buildpack runs more like other languages buildpack like Ruby and NodeJS.

Note that I consider this a beta, and you have to specify the git branch #carton in the buildpack URL to use it.

The initial deployment takes a little while to copy vendor perl, then bootstrap all dependencies for carton - which i hope can be eliminated once carton goes 1.0 and support fatpacked executable.

Thanks to @judofyr for his also excellent perloku buildpack that i shamelessly copied the technique to build & download perl binaries.

Jun 18, 2013
Make LWP optional. by miyagawa · Pull Request #408 · plack/Plack → github.com

This eliminates the XS dependency in Plack.

Plack-1.0026-TRIAL is on CPAN, which is a big release that eliminated a dependency for the big LWP module.

LWP not only has a big chain of dependencies since the distribution split earlier, it also has XS dependency (HTML::Parser) in the chain, which makes Plack code non-trivial to install on environments without a C compiler, and makes fatpacking difficult (although realistically, these C bits are only necessary for testing and not required mostly on the runtime).

This is a big shift - I made LWP optional in #408 but also replaced the use of LWP with HTTP::Tiny in #410 - so that Plack::Test::Suite (reference Test suite for PSGI handlers) and ::Server (Plack::Test implementation to run tests through live server) now uses HTTP::Tiny instead of LWP. The test interface is still the same, and you can use HTTP::Request/Response interface for that, which is a much smaller subset of LWP.

I tested various distributions where it implicitly expects LWP when depending on Plack, and fixed a lot of them, and sent pull requests where needed. Please test Plack-1.0026-TRIAL against your application/frameworks/server implementations.

cpanm --dev Plack
Jun 13, 20132 notes
Podcast streaming howto

WWDC podcast としては日本最速(たぶん)のepisode 13 で3度めになった ライブ配信 について簡単に解説。

先日 RubyKaigi なんかでいろんな人から「Podcast 聴いてます!」といってもらえたのはうれしかったんですが、ある人から「どこで録音してるんですか?」と聞かれました。「家ですね」と答えたら、「え、じゃあゲストの人を家に呼んでるんですか?」と返ってきたのはちょっとおもしろかったです。

配信・録音は Skype でやっています。日米で離れていることもよくあって、スケジューリングが割りと大変。収録自体は Skype Call Recorder などでそのまま録音もできます(実際自分の声はこれで録音しています)が、クオリティを上げるためにローカルでも録音したものをマージしています。この辺の話は Podcast Recording and Editing に書きました。

で、ライブ配信をするにあたって、単に Skype 上で流れている音を配信するなら Soundflower, Line-In, Garage Band なんかを駆使すれば できそうです。が、

  • Skype と配信で(ただでさえ細い)自宅の帯域を占有したくない
  • 複数の重いアプリを立ち上げてCPUを駆使したくない。ファンが回るとマイクで拾ってしまうこともある

という理由から、別途配信用のマシンを日本に用意しています。用意しています、といってもすでにテレビ録画などに使っているMac miniマシンが日本にあるので、これを使ってます。

配信用の Skype bot アカウントを作成し、実際に収録時にボイスチャットに参加させます。contact オンリーにしておいて、自動でaccept するようにしておくと便利ですね。操作は LogMeIn から。

Skype 側では、流れてくる音を Soundflower (2ch) にルーティングします。もし自分のSkype でやるとなると、自分の声を Line In などでミックスする必要がでてきますが、これは bot で聴く専門なので、必要なし。

あとは配信用のアプリに Soundflower 2ch を入力として入れればOK。ライブ配信に使っている Mixlr では入力ソースをカスタマイズできるので、ここで選択すればOK。配信スケジュールの設定や iOS アプリからの視聴からもできていいですね。

ライブページ に使っている埋め込みウィジェットは有料会員のみのようですが、同様の他サービスに比べてリーズナブルなので、とりあえず有料にしてみています。

ライブ配信のスケジュールはだいぶ不定期ですが、iCal フィード を google calendar 上に用意したので、リアルタイムにツッコミを入れたい、という場合にはこちらも利用してみてください。

Jun 11, 20137 notes
Tatsuhiko Miyagawa's Podcast: ep13: WWDC 2013 (Kenn Ejima, Hakuro Matsuda) → podcast.bulknews.net

bulknews-podcast:

収録時間 46:55 | Download MP3 (26MB)

Kenn Ejima さん (@kenn), Hakuro Matsuda さん (@hak) を迎えて、Apple WWDC 2013 Keynote, iOS 7, OS X Mavericks, Mac Pro, Macbook Air などについて話しました。

3人収録にも慣れてきたこの頃、おなじみの kenn と、hakuro さんを迎えて WWDC 話です。収録ではさらっと流しているというか触れてもいませんが、hakuro さんは元 Apple で iOS のエンジニアリングをしていたこともあり、なかなか濃い話ができました。

ちなみにゲスト2人の場合、個人的には話すネタに困ることが少ないので事前の準備は楽だったりしますね。オーディオ漏れが起こりやすかったり、ぶつかることも結構あるので 編集が割と大変 だったりしますが、だいたい収録時間 x 3 ぐらいで作業できるようになってきました。

Jun 11, 201310 notes

May 2013

6 posts

Paying respect to Module::Build | David Golden → dagolden.com

I want to take a moment to discuss what this means, why I suggested it, and what I think Module::Build trail blazed for Perl 5.

Great post (as always) by David Golden about what Module::Build pioneered since 2001 to make the Perl toolchain ecosystem better.

Few things to add:

So… the biggest impact on end-users will be that when Perl 5 version 20 is released in Q2 2014, people will have to install Module::Build from CPAN to squelch the deprecation warning.

This is usually automatic, since major CPAN clients figure out whether the module loaded from perl core lib dir is marked deprecated, and in that case it automatically reinstalls it from CPAN, and the deprecation warning will go away.

If you had a subclass to do one thing and I had a subclass that did something else and you wanted to combine them, you pretty much had to copy and paste code.

I should point out there’s a work to combat this problem: Module::Build::Pluggable by tokuhirom. But I should also note that tokuhirom and I eventually agreed that most of the authoring side work should be done by the author tools, hence we are now working on Milla and Minilla - it’s no coincidence we have similar names.

May 28, 2013
Tatsuhiko Miyagawa's Podcast: ep12: Perl 5.18, YAPC::Asia (lestrrat, tokuhirom) → podcast.bulknews.net

bulknews-podcast:

牧大輔さん (@lestrrat), Tokuhiro Matsuno さん (@tokuhirom) を迎えて、Perl 5.18, YAPC::Asia などについて話しました。

May 27, 20136 notes
Tatsuhiko Miyagawa's Podcast: ep11: Google I/O 2013 (kenn, mootoh) → podcast.bulknews.net

bulknews-podcast:

収録時間 46:22 | Download MP3 (26MB)

Kenn Ejima さん (@kenn), Motohiro Takayama さん (@mootoh) を迎えて、Google I/O, Google Glass, Google Play Game Services などについて話しました。

番組へのフィードバックは Twitter にて @miyagawa またはハッシュタグ #bulknews にてお寄せください。

Show Notes
  • Google IO
  • The best of Google I/O 2013…

2週間で3エピソードとちょっと飛ばし気味ですが、時事ネタということでGoogle I/O 話。翌日でキーノートちょっと忘れてますが、参加してたmootohさんのおかげでいろいろ拾えましたね。後半の Game Services 話は業界トークっぽくて面白かったです。

録音しながらライブストリーミングもしてみたんですが、時間帯の関係もあってフィードバックをもらうのが難しかったですね。twitter 以外にも Lingr とかのほうがエンゲージしやすいのかな。いろいろ試行錯誤してみる必要ありですね。

May 17, 201311 notes
Podcast recording and editing

Here’s what it looks like when it comes to editing my (semi-badly recorded) Podcast episodes.

When I first heard 5by5/Mule’s podcasts, i was really impressed by the audio quality of their shows, since like many of you, i had a feeling that “Podcast is by amateurs and audio is mediocre at best”. I was blown away.

So I did a little bit of research before starting my show, because I wanted to make it sound good too. Jason Snell’s article on MacWorld explains the current method of my recording.

Skype, Everyone Records, GarageBand

The tl;dr for that is, I use Skype Call Recorder to record both my audio and guest(s) on Skype, on separate tracks. It’s great that it records my audio locally, not via Skype, so it’s a direct input from my Mic.

I ask guests to record their local audio using QuickTime too and send files after recording. Then I synchronize them with Skype’s recorded audio, then discard the Skype audio file.

Now I get everyone’s locally recorded audio, on separate tracks on GarageBand, and I only need to edit when there’re murmuring or conflicts, etc. In Theory.

I’m often asked how long it takes to edit the show - if the show is one hour, it usually takes 2 to 3 hours.

Microphone and Headphones

Recent episodes needed a little bit more of my editing work than previously - this is not to particularly blame anyone but the technologies/hardware, because it’s a mere audio leak on other’s ends.

Basically when I speak, my voice leaks on the guest’s track because their mic picks up the audio from the earpiece, and makes an echo/delay sound effect that I need to cancel out. That leads to the chaotic GarageBand editing shown earlier. Noise gate works to cancel that, but it also cancels the valid voice and i avoid doing it.

I ask everyone to use headphone/earphone, but sometimes the microphone is too close to them, or the headphone is not solid enough and leak audio from there. It doesn’t seem to happen to everyone (which is good), so it should definitely be related to the combination/setup of the microphone and headphones.

My particular setup is Blue Yeti USB Microphone and Klipsch IMAGE S4 and don’t have that problem on my end. Klipsch has a very crisp sound with very little sound leak, as an in-ear headphone.

In terms of not leaking audio, Over-the-ear Headphones should be the best, but unless you have a microphone with monitoring capabilities (like Yeti :D), it might be difficult to hear what you speak while speaking.

I guess this is one of the difficulties of doing the show with irregular guests, since we can’t ask guests to invest some money on microphones/headphones, which I would if I do recurring/fixed guests :)

Tips

Here’s a simple thing: don’t use Apple’s iPhone headset. Its audio isn’t bad, but it picks up a lot from its earpiece, and makes a big noise when touched with your clothes. Speaking of Apple, Macbook Pro Retina’s dual USB microphone is fantastic, assuming you don’t have a big ambient noise.

Also, some of the Japanese podcast shows I listen to have severe audio quality issues (although I’m not sure if they read this in English ;p) - Just don’t record the show in a cafe/restaurant with big ambient noise. Also, double your bitrate and make a mono MP3, rather than low bitrate stereo files.

—

Anyway this was my current setup. For the latest episode 11 where we talked about Google I/O, we also experimented with the live streaming with Mixlr. I will explain the setup about it (pretty simple if you have a spare Mac mini) later if you’re interested :)

May 17, 2013
Tatsuhiko Miyagawa's Podcast: ep10: RailsConf, Ruby 2.0, Rails 4 (mrkn, r7kamura) → podcast.bulknews.net

bulknews-podcast:

Kenta Murataさん (@mrkn), Ryo Nakamuraさん (@r7kamura) を迎えて、RailsConf, Ruby 2.0, Rails 4, Chanko 2.0, RubyKaigi, YAPC などについて話しました。

今週は日本語営業に戻って、mrkn, r7akamura とRuby 話をしました。ちなみに3人の収録だと間が持ちやすくていいんですが、編集がちょっと大変。

Rails 2.3 is Kansas

DHH の Kansas ネタはConfreaksにKeynoteが上がってました。13分あたり。hkmurakami さんが 詳しく解説 してくれてます。日本の2ch でよくでる群馬県ネタ に近いものがありますね。

May 12, 20139 notes
Tatsuhiko Miyagawa's Podcast → podcast.bulknews.net

bulknews-podcast:

On the episode 9 of my podcast, featuring Jesse Vincent (@obra), we discussed his recent experiments to build his own keyboard.

Some of my followers on Twitter and friends on Facebook asked if there was any plan for an English episode of my podcast.

So here it is, with Jesse Vincent, we talked about his recent exploration of making his own Keyboard. We also discussed little bit about perl, and OSDC.TW, the conference we just attended a couple of weeks ago.

Might want to do more of English episodes in the future, but comments and feedback is always appreciated.

May 8, 20138 notes

April 2013

7 posts

Starman 0.3011 → metacpan.org
  • Disabled flock serialization when it’s unnecessary. This will improve the performance when you have many Starman worker processes on Linux systems (kazeburo) #69

It’s a minor version update, but contains very important fix - it bypasses flock serialization in accept in most UNIX systems where there’s only one port being listened. It will boost the performance when you have a bunch of worker processes. Thanks to kazeburo for the patch on his blog!

Apr 24, 20132 notes
Freeze CPAN deps with PAUSE git → github.com

PAUSE is the backend of CPAN module upload server, and since April 2012, thanks to rjbs, for every perl module upload, it commits the index list (as well as author/permission mapping). And now it is available on github. Thanks Andreas!

This makes it so easy to freeze CPAN module dependencies for your application solely based on time.

First, check out the repo, and then specify the revision with date time you want to freeze at:

git clone git://github.com/batchpause/PAUSE-git.git
cd PAUSE-git
git checkout `git rev-list -n 1 --before "2013-01-01 00:00" master`

Then, refernce that 02packages from cpanm:

cpanm --installdeps . \
  --mirror-index /path/to/PAUSE-git/02packages.details.txt
  --mirror http://backpan.perl.org/

That’s it, really.

If you want more control over versions, dev releases and git forks, you might need Carton. And you can only go back to 2012 April and no older than that. But for a simple time-based index, this is all you need.

Apr 24, 2013
kazeburo/Monoceros · GitHub → github.com

Monoceros is PSGI/Plack server supports HTTP/1.0. Monoceros has a event-driven connection manager and preforking workers. Monoceros can keep large amount of connection at minimal processes

Great addition to the PSGI/Plack web servers. Gets the best of event-based servers to handle C10K connections while working a pool of preforking workers.

Apr 24, 20132 notes
nginx 1.4.0 is out → nginx.org

nginx-1.4.0 stable version has been released, incorporating many new features developed in the 1.3.x branch - support for proxying of WebSocket connections, OCSP stapling, SPDY module, gunzip filter and more.

Apr 24, 20131 note
Listen-IT: 3編セット40%オフキャンペーン中 → blog.listen-it.com

宮川達彦さんのPodcastをスポンサーしましたので、その記念に40%オフキャンペーン開催中です。母音編、子音編、文章編、3つセットでご購入いただくと定価2000円なのですが、今なら1200円。全部で60レッスン、669単語と170の文章が詰まっていますので、1レッスン20円、単語/文章当たり1.4円と大変お買い得です!

Great app, 40% off sale. Go buy it and support my podcast :)

Apr 17, 20131 note
Tatsuhiko Miyagawa's Podcast: Podcast ep8: ゲスト 伊藤直也 高林哲 → podcast.bulknews.net

bulknews-podcast:

スポンサー: Listen-IT

伊藤直也さん (@naoya_ito) 高林哲さん (bkノート) をゲストに迎えて、日米キャリアパス、スクリーンキャスト、ペアプログラミング、エンジニアリング英語などについて話しました。

おなじみのお2人と、都内某所で収録しました。後ろで将棋を指してる音が聞こえます :)

本記事にもありますが、収録機材がノイズを結構拾ってしまっていて、ちょっと気になるかもしれませんがご了承を。やっぱりマイクのクオリティが音質に直結しますね。Skypeだとあとで編集はいろいろできるんですが、直録りはいろいろ難しいです。

このあとまだまだたくさんしゃべったんですが、長くなったのでここまで。またお蔵出しもあるかもです。

Apr 17, 201313 notes
Tatsuhiko Miyagawa's Podcast: ep7 ゲスト: ゆーすけべー (yusukebe) → podcast.bulknews.net

bulknews-podcast:

収録時間 40:06 | Download MP3 (24MB)

第7回は和田裕介さん (@yusukebe) をゲストに迎えて、Perl プログラミング、フレームワーク、モジュール開発、YAPC などについて話しました。

いつになく、Perl成分の濃い話になりました。最近、昔話多め感もありますが、それもアリということで。

後半の方ででてきた、Fat vs Thin libraries の話は、前回のYAPC::Asia のパネルディスカッションでも同じような話をしたんですが、YAPCでのスピーカー勢の(大半の)考え方と、実際に使われているモジュール・ライブラリ間でギャップがあったりしないかなぁ、という思いを最近持っていて、ネタにしてみました。

軽いものが使いやすいのはそうなんですが、重いもの、依存が多いものにはそれなりの理由があったり、枯れていたりというメリットもあるわけで、適材適所かなぁ、と最近は思っています。ま、同じ事ができて速いなら、そっちのほうがいいには決まってるんですけどね。

Apr 3, 201313 notes

March 2013

7 posts

test_requires all the way

tl;dr Today possibly for the first time ever, CPAN toolchain ecosystem all support test requirements as separate from build requirements. I can’t be happier ever.

Little bit of history. Timeline might be not in the correct order.

Module::Install always had test_requires() with forward compatibility in mind. However META spec v1.4 didn’t support specifying test requirements separate from build requirements. So the most tools merge them as a build requirement, including Module::Install itself. This means most of the Test:: module dependencies, even though they are only used for testing, were installed even when notest options is supplied (with CPAN.pm and cpanminus).

META spec v2.0 came out, and it allows META (and MYMETA) to specify test requirements separately.

cpanm supported parsing test requirements separately, and skips installing test prereqs when —notest is in use. But at that time, most of the tools did not support emitting test requires apart from build requires.

Except Module::Build::Tiny, which just copies META.json to MYMETA.json upon installation, so if you use an authoring tool such as Dist::Zilla or Milla to emit META.json with test requirements apart, it will be skipped.

Meanwhile rjbs added TEST_REQUIRES support to MakeMaker at Paris QA hackathon 2012. 7 months later MakeMaker 6.63_03 was released with the fix, and non-dev version was just out in December.

Same fix was added to Module::Build by tokuhirom and released as 0.40004 today.

Now, what should happen is to update Makefile.PL and Build.PL to make use of these new features - I patched Dist::Zilla so that MakeMaker and ModuleBuild emitters will use test_requires when the installer has the version with the support for it.

I’m trying to fix Module::Install similarly, but as of this writing I’m not that hopeful - if someone wants to take over my work to make it merged that’d be super. But at this point I’d rather give up and stop using Module::Install.

Anyway, now when you have a module that is only used for testing, it will be (and has to be) declared and emitted correctly as a test dependency, and when you run CPAN installers, at least cpanm, with “no test” option, they won’t be installed. This is a very important milestone.

Mar 30, 20131 note
Milla, a Dist::Zilla profile that doesn't suck (screencast)

Ok, that was a little link bait title.

I’ve been a big fan of Module::Install - it automatically figures out the metadata of my module with just all_from, and does the right thing creating META files. Then it bundles itself in inc/ so that if you install from CPAN you don’t need to have Module::Install pre-installed.

But not everybody likes it, especially contributors who need to install plugins before doing anything, or users who want to grab the latest from git.

The problem is, Module::Install is just a wrapper for MakeMaker, and MakeMaker and Module::Build are not the right tool to create a distribution.

Dist::Zilla is a pioneer in this field, to provide the complete authoring environment for the CPAN distribution, while having the ability to export MakeMaker/Module::Build compatible build files, so that end users don’t need to have that upon installation.

However Dist::Zilla kind of sucks. Well, that was a strong word - it has been coded and maintained actively, so it’s great. But its user experience could have a little improvements.

Milla is my first attempt to address that issue. It is a set of profile - PluginBundle, MintingProfile and command line wrapper, so that you can use it with almost zero-configuration and should do the right thing.

There’s some limitation and assumption that do not work for everybody, but at least for me I hope to replace 95% of my CPAN modules with Milla in the coming weeks.

Here’s a screencast to show you how to use it.

If you already have modules written with MakeMaker/Module::Install + ShipIt, see the Migration document for the details. I might make a new screencast just for the migration.

Hope you enjoy Milla. If you dislike the 195 dependencies it needs, check out our sister project Minilla as well.

Mar 27, 20131 note
Tatsuhiko Miyagawa's Podcast: ep6 ゲスト: Naoya Ito → podcast.bulknews.net

bulknews-podcast:

収録時間 41:20 | Download MP3 (23.5MB)

第6回は伊藤直也さん (@naoya_ito) をゲストに迎えて、Kindle 出版、GitHub、Google Reader などについて話しました。

第1回に続き、naoyaさんに出てもらいました。

「非公式なAPI」

後半で出てくる Google Reader APIの話で、「野良API」って言葉を使ってますが、紛らわしい表現だったかもなので、ちょっと補足。

「野良API」というと第三者が無断でつくったAPI、のようにも聞こえますが、そうではなく、「非公式なプライベートAPI」という方が正しい。

ここでいう非公式なAPI、というのは、Google Reader のウェブサイトが Ajax で使っているエンドポイントや、Google公式の Reader アプリが使っているプライベートAPIをリバースエンジニアリングしたAPI、という意味で使っています。仕様についても Google は一度も公開したことはなく、有志がリバースエンジニアリングした仕様をGoogle Code上で公開 してるもので、これを feedly などは使っていた、という話です。

ゲストの人選

naoyaさんが2回めということもあって、「もう2周目」?とかTwitter で意見をもらったりしてます。

もともと、ゲストを週替りにしてインタビューする、という発想ではなくて、その時話したい「ネタ」があって、それに最適な人をキャスティングする、という発想をベースにしていました。MessagePack のissueの話、Ruby 2.0 のリリースの話、その開発者に聞くのが一番いいですよね :)

とはいえ、ネタにとらわれずに「いいとも」あるいは「徹子の部屋」的に人にフォーカスした回もあってもいいかなと思っていますし、ep4の高林さん の回なんかはそういう感じですね。

その辺もいろいろバランスをとってやって行きたいと思いますが、まだまだいろいろ試行錯誤ですので、フィードバックもらえるとうれしいです!

Mar 21, 201321 notes
Next page →
2012 2013
  • January 1
  • February 12
  • March 7
  • April 7
  • May 6
  • June 4
  • July
  • August
  • September
  • October
  • November
  • December
2011 2012 2013
  • January 17
  • February
  • March 2
  • April 2
  • May 5
  • June 2
  • July 1
  • August 7
  • September 2
  • October 5
  • November 6
  • December
2011 2012
  • January
  • February
  • March
  • April
  • May
  • June
  • July
  • August
  • September
  • October
  • November 6
  • December 17