22 6 / 2014

ghq + peco/percol

This has been rocking my Twitter developer community for the past few days, but mostly only in Japanese - here’s an attempt #1 to fix this.

tl;dr ghq allows you to organize git clones via a simple CLI and peco or percol makes cd’ing to these directories on your shell a snap.

GOPATH/src for everything

Go has an interesting directory structure that forces you to adopt when you write your own Go program. All your code, along with third party libraries, will be placed in the src/ subdirectory under the $GOPATH directory with each hostname beneath that. It’s a little weird at first but when you get used to it, it makes a lot of sense.

On a recent episode of Rebuild.fm my guest lestrrat mentioned that he adopted this directory structure to all the code he writes, including non-Go code.

I think this is a good idea. And I now set:

export GOPATH=$HOME

so all the code is under ~/src.

ghq to manage repos

Now we’re going to manage these github repositories under ~/src, but doing so manually is a pain because you have to make subdirectories (like github.com/) for each. ghq makes that easy.

By default ghq clones the repo under ~/.ghq but i prefer it to be the same value as the one we specified earlier:

git config --global ghq.root ~/src

Now you can clone repo from github easily, i.e.

ghq get plack/Plack
ghq get git://mygithost.example.com/project.git

and it will be cloned into ~/src/github.com/plack/Plack etc.

cd to these repos

Now we got plenty of repos under ~/src, and when you want to cd into one of these directories, that’d be a pain, because of the deeply nested directory structure.

peco to the rescue.

  • ghq list -p will print out all the repos in full path
  • peco gives an interactive UI to incrementally filter the directories

Combining the two, all you need to do is:

> cd $(ghq list -p | peco)

and type some query to match the repo you want, then hit enter.

That command is annoying to type - here’s my zsh setup so that I can hit Ctrl-] on my shell to bring up that UI to cd to.

function peco-src () {
    local selected_dir=$(ghq list --full-path | peco --query "$LBUFFER")
    if [ -n "$selected_dir" ]; then
        BUFFER="cd ${selected_dir}"
        zle accept-line
    fi
    zle clear-screen
}
zle -N peco-src
bindkey '^]' peco-src

Importing CPAN/Rubygem libraries

You can clone git repo with ghq if you know the github URL etc. But what if you have a CPAN/Rubygem that you want to fix and don’t know which repo? There are tools for that.

cpan-ghq allows you to pass git repos (in META.json) to ghq, and gem-src provides a gem plugin to run ghq for each install of Gems.

Permalink 15 notes

15 6 / 2014

Goro Fujiさんとポッドキャスト販売などについて話しました。

Rebuild の収録後に録音している Aftershow こと「終了後のお楽しみ」コーナーを配信開始しました。

当初は、これだけ集めて別フィードにしようかとか、有料配信にしようとかいろいろ考えてましたが、面倒だし通常通り聞いてもらいたいので、フィードにそのままミックスして配信しています。

毎回、10分程度とったものをノーカットで、本編配信の数日後に出す形でやっていこうかなと思います。本編じゃ物足りない、って人も軽く聞いてもらえるといいなと。フィードバックもお待ちしています!

Permalink 4 notes

09 6 / 2014

伊藤直也さんをゲストに迎えて、GitHub, Pull Request, Hubot, リモートワークなどについて話しました。(6/1 GitHub Kaigiにて収録)

Github Kaigi で公開収録したエピソードを公開しました。500人と、すごい規模のお客さんでしたが、ラジオ番組の公開収録ってこんな感じだよな〜とおもいながら収録してました。自前で持ち込んだ機材(会場、GitHubストリーミングとあわせて各自マイクx3の記者会見仕様)で録音しましたが、それなりに聞きやすくとれてるとおもいます。基本的には完全ノーカットでお送りしています。

当日いい忘れてしまいましたが、オーガナイズに関わったスタッフのみなさん、githubやサイバーエージェントをはじめとするスポンサーの皆様、ありがとうございました。

Permalink 4 notes

08 6 / 2014

OS X automation will get a whole lot easier with JavaScript. No need to look up AppleScript cheatsheet every time i need to script something.

Permalink 4 notes

08 6 / 2014

Weʼre still not allowed to post screen shots (the rumor sites will do that anyway) and I am not sure what qualifies as a “public review”. Am I not allowed to post my opinion about a new API or is that solely meant to prevent full reviews of the entire OS before the public release?

Assuming that talking on podcasts doesn’t qualify as “write public reviews”, this is a good news :)

17 5 / 2014

Our own Jafar Husain shared how we’re using the Reactive Extensions (Rx) library to build responsive UIs across our device experiences.

Pretty impressive talk on Rx by Netflix. Looks like a great alternative to Promises, on handling asynchronous behaviors such as remote API calls and even UI elements.

Permalink 2 notes

17 5 / 2014

Great post on Python 3 and Unicode (and UNIX).

The much more likely thing to happen is that people stick to Python 2 or build broken stuff on Python 3. Or they go with Go. Which uses an even simpler model than Python 2: everything is a byte string. The assumed encoding is UTF-8. End of the story.

Hopefully there’s a lot to learn from this post on the danger of defaulting everything to UTF-8 and forcing them. Assuming utf-8 by default is another story, and I find that Golang is doing this very well.

11 5 / 2014

Rebuild.fm Live push notifications

Rebuild のライブ配信スケジュールはカレンダーやTwitterでお知らせしています。配信開始の時間にモバイル端末でプッシュ通知を受け取る方法をいくつか紹介します。

Twitter アプリの通知を有効にする

Twitter のオフィシャルアプリ iOS / Google Play がインストールされている場合、@rebuildfm をフォローして push 通知を有効にすることができます。ライブ配信の30分前や開始時にお知らせしています。

アプリから @rebuildfm アカウントを開き、設定の「通知を有効にする」を選択すればOK。

ちなみにアカウントによっては、このオプションを有効にすると初期にフォローしたアカウントのほとんどでプッシュが有効になるバグがあるようです。アプリの設定から、Notifications > Tweets でリストが管理できます。

カレンダーを登録してアラート設定する

Live ページ に iCal と Google Calendar のリンクがあるので、これを iOSカレンダーやGoogle Calendar に登録します。

デフォルトではアラート設定されていないので、システム設定から、カレンダー>デフォルトアラート設定を指定します。

iOS でカレンダー単位のデフォルトアラート設定する方法が見つからなかったのですが、他に方法があれば教えて下さい。

スクリーンショットではiOS での場合を紹介しましたが Android でも Google Calendar や Twitter 公式アプリを使って同様に設定可能です。

Permalink 9 notes

08 5 / 2014

App Links

App Links - Link to what you want, wherever you are.

Publishing App Link metadata is as simple as adding a few lines to the tag in the HTML for your content. Apps that link to your content can then use this metadata to deep-link into your app, take users to an app store to download the app, or take them directly to the web to view the content. This allows developers to provide the best possible experience for their users when linking to their content.

This looks a lot like Mobile Link Discovery i published back in 2005, to support mobile-device compatible URL (this was before iPhone) in meta tags, so that search engines can link to the mobile friendly URL before even linking to it. Google Japan’s mobile search supported it.

Twitter does App Cards with its Twitter card as well, interesting to see where this goes, although i haven’t seen much adoption of it, given the Twitter card is only supported by Twitter native apps yet.

07 5 / 2014

Covert Redirect Vulnerability with OAuth 2

tl;dr Covert Redirect Vulnerability is a real, if not new, threat when combined with Implicit Grant Flow (not Code flow)

This Covert Redirect Vulnerability in OAuth 2 is an interesting one.

There’s a couple of defending arguments that this isn’t a flaw in OAuth itself.

While I agree that it isn’t a flaw in the protocol, I think the threat is a real one, combined with a) a loose validation on redirect_uri on the OAuth provider and b) an open redirector on the client site.

What seems odd to me in the defending arguments is that they mostly focus on the “code” response_type, and it’s argued that the attack is unlikely because a) obtaining a code doesn’t complete the attack and b) the open redirector needs to keep the query parameter, which is crazy.

They’re both correct.

However what needs to be addressed here is the Implicit Grant flow, where a token is immediately sent to the client redirect URI using the URI fragment.

When redirect_uri parameter can be set to an open redirector (like Amazon/ESPN case), the token is sent to it this way:

http://provider.example/oauth?
  response_type=token&
  client_id=xyz&
  redirect_uri=http://client.example/redir%3fto=http...

which goes to:

http://client.example/redir?
  to=http://malicious.example/
  #access_token=xyz

People tend to believe that the URI fragments will be thrown away in the next redirect:

So that is all true, where it gets strange is that the browser he is using seems to pass the URI fragment in the URI through the redirector. This is not possible if the browser has not been tampered with.

It is possible with the standard behavior of the browsers.

With my experiments, current Chrome and Firefox will keep that URI fragments and access the malicious site with the fragment. W3C document suggests that sending it is the desired behavior. Safari doesn’t seem to send it, and it’s considered a bug. I haven’t tested but MS Blog suggests that IE10 preserves the fragment, matching “the updated standard documents”.

So, the victim is now redirected to http://malicious.example/#access_token=xyz.

Now, the fragment isn’t sent to the malicious site over the wire as part of an HTTP request, but once the browser gets to the page it’s game over - the site can grab the token using JavaScript via location.hash pretty easily.

Having said that, the right answer to this threat would be:

  • Strict validation on redirect_uri
  • Do not implement an open redirector on the client domain

and RFC6819 has already covered those as well.

Yes this is a FB/ESPN/Amazon problem, and No, this isn’t a design flaw in OAuth 2.0 - however, the defense argument saying “this attack is unlikely because getting a code doesn’t complete attacks” or “tokens won’t be preserved during redirects in regular browsers” is a wrong one.

Permalink 5 notes