YouTube video offline mode

Recently I enrolled to AI class online course. Each week they submit dozen of explanatory and Q&A videos on youtube and reference them from AI-class site. As I mostly have time to study those videos when I'm offline I've decided to explore possibilities of getting videos to local computer for offline viewing. Purpose of this post is just to summarize steps which I did in order to achieve this goal on linux.


Get tools

First thing first, I had to equip myself with tools.

youtube-dl
After unsuccessful attempt to use clive/cclive, I decided to use youtube-dl which you can download at github project page. Although it is single purpose youtube only downloader it works and does exactly what I need. You need python interpreter to run it.
grake
Grake is utility to parse youtube links out of HTML and it's hosted at code.google.com. Grake is Perl module, but installation is straightforward, as one only has to follow instructions in INSTALL file.


Get list of videos
First step is using youtube.com to find playlist of videos. For example I've just typed 'AI class unit9 videos' to google/youtube to get this link: 'http://www.youtube.com/playlist?list=PL9163DC3C43AF7612'.


Next step is to download list using Grake and edit it

$ cd ~/Videos/Unit9
$ grake http://www.youtube.com/playlist?list=PL9163DC3C43AF7612 < Unit9.lst

Downloaded file looks like:

http://youtube.com/watch?v=videoseries
http://youtube.com/watch?v=DgH6NaJHfVQ
http://youtube.com/watch?v=9D35JSWSJAg
http://youtube.com/watch?v=9QMZQkKuYjo
http://youtube.com/watch?v=YfSBYf9h7qk
...

I've just deleted first line using vim.


Get videos
Last step is to get the videos by using youtube-dl. I've used this:

$ cat Unit9.lst | xargs youtube-dl -tA

With -tA options I told downloader to use video name as filename instead of funky hash and autoprefix it with number as it's downloaded.

Groovy Junit tests in Eclipse Indigo

I'm playing with Groovy and Grails a little bit and recently I had a problem running unit-test from recent Eclipse Indigo (3.7) with groovy-eclipse plugin installed.

Problem manifests itself with following error, when I try to righ-click and "run as" junit test-case.

junit.framework.AssertionFailedError: No tests found in path.to.your.class

I've spent some time trying to find solution and it was obvious. Think it might be helpful to keep solution in some form. I'll probably post more Groovy/Grails related stuff in the future, so why not start with this.

Let's have simple class MyClass

package sk.adino.poc.groovy.unittests

class MyClass {
  x() { return 1;}
}

and test class MyClassTest in same package

package sk.adino.poc.groovy.unittests

class MyClassTest extends GroovyTestCase {
  testX { assert x==1 }
}

Right-click in eclipse and run as JUnit test should work, but you have to:

1. have groovy-eclipse plugin installed
2. add external JAR pointing to junit-x.y.z.jar from your groovy installation lib directory - otherwise it won't even compile
3. add void as a return type of your testX function so it should look like:

package sk.adino.poc.groovy.unittests

class MyClassTest extends GroovyTestCase {
  void testX { assert x==1 }
}

Java puzzler - Letter went missing

Recently I've implemented function that takes String and populates long so that it contains character values from the back of the string. However my unit test is failing because one Letter is missing. Are you able to put it to the right place so that unit tests will not fail without debugging the code?

import static org.junit.Assert.*;
import org.junit.Test;

public class StringToolsTest {

	static final char [] data = {0xca, 0xfe, 0xeb, 0xab, 0xee};
	static final String string5 = String.valueOf(data);
	static final String string0 = "";
	static final String string1 = "a";
	static final String string9 = "987654321";
	
	@Test
	public void testFillBackwards() {
		long l = 0;
		assertEquals(0xcafeebabeeL, StringTools.fillBackwards(string5, l));
		assertEquals(0L, StringTools.fillBackwards(string0, l));
		assertEquals(0x61L, StringTools.fillBackwards(string1, l));
		assertEquals(0x3837363534333231L, StringTools.fillBackwards(string9, l));
	}
}

public class StringTools {
	public static long fillBackwards(String s, long d)
	{
		long result = 0L;
		byte[] bytes = s.getBytes();
		for (int i=0; i<Math.min(8,bytes.length); i++){
			result |= ((bytes[bytes.length-i-1] & 0xff) << (8*i));
		}
		return result;
	}
}

Solution bellow (highlight to show):

The problem is on the line 29 in "& 0xFF". I should be doing &0xFFL instead. Why ? bytes[x]&0xff is by default int (both operands are widened to int), but then << is applied, result of which is type of left operand (java spec) and this is |= to long result. Solution is to make & operator to work on long values by adding L to 0xFF.

Puzzler on snipplr.

Question Interview

One of my friends got this question on his interview: "Given is an arbitrary long string, revert order of words in the string". If the input is "Hello World!", function should modify string to: "World! Hello". Function must have this interface: "void revert(char * string)". It must work in linear time O(n) and have no special memory requirements M(1).

My solution (spoiler alert)

Java puzzler - How many integers are in the box ?

This is most basis Java Puzzler. Question is, how will the output look like ?

• Will it print out powers of 10 starting from 1 ending at 10000000.
• Or will it print just 1.
• Or will it print any other number.
• Or will it loop forever.
• Or anything else.

package sk.adino.puzzlers;

import static org.junit.Assert.*;
import org.junit.Test;

public class AutoboxingTest {

	@Test
	public void test() {
		int i=1;
		while(i<10000000) {
			System.out.println("value:" + i);
			Integer j=i;
			Integer k=i;
			if (j==k){
				i=i*10;
			}else break;
		}
		assertEquals(i,1000);
	}
}

Puzzler @ Snipplr

Puzzler for C developer role

Today, I've created puzzler for C developer at company I work for. This is one of rare cases where we can publish the code, and it's few lines anyhow, so here I give it a shot. You can try to solve the puzzler, as described in comments. It's very easy and you should have no problem solving it without Solaris. All you need is a C compiler.

Bonus question: why did I choose (R) and (C) instead of R and C ?

Bellow is the code which you can find also on snipplr:

1. /** This program should be compiled using following command:
3.  $ cc -g -o test_hr_test test_hr_test.c
5.  Original developer Malvin left company to pursue his dream and went
6.  to live with wild ligers. No-one ever heard any word about him since
7.  he left. We've managed to recover source code from his source repo.,
8.  but program seems to be broken. Strangely Malvin used iso-8859-1 as
9.  his console encoding.
11.  Hints:
12.  - program does not compile
13.  - we were not able to recover whole function permutateBlockRev()
14.  - program crashes on our X86 solaris 10 matchine (not on linux)
16.  Are you the one which can help us recover the password ?
17.  Send it to: rpc-hr@ri-rpc.sk
18.  */
20. #include <stdio.h>
21. #include <string.h>
23. /** bit manipulation macros */
24. #define CLRBIT( STR, IDX ) ( (STR)[(IDX)/8] &= ~(0x01 << (7 - ((IDX)%8))) )
25. #define SETBIT( STR, IDX ) ( (STR)[(IDX)/8] |= (0x01 << (7 - ((IDX)%8))) )
26. #define GETBIT( STR, IDX ) (( ((STR)[(IDX)/8]) >> (7 - ((IDX)%8)) ) & 0x01)
28. /** permutation works on unsigned char blocks of size 8 */
29. #define BLOCK_SIZE 8
30. typedef unsigned char[BLOCK_SIZE] block;
32. /** Function does reverse permutation to DES initial block permutation as described at
33.  http://en.wikipedia.org/wiki/DES_supplementary_material
34.  @param b - block which will be permutated in-place
35.  @note indexes start from 0 contrary to wikipedia initialization vector - C like index
36.  */
37. void permutateBlockRev(block b)
38. {
39. int i;
40. static const int initialPermuteMap[64] =
41. {
42. 57, 49, 41, 33, 25, 17, 9, 1,
43. 59, 51, 43, 35, 27, 19, 11, 3,
44. 61, 53, 45, 37, 29, 21, 13, 5,
45. 63, 55, 47, 39, 31, 23, 15, 7,
46. 56, 48, 40, 32, 24, 16, 8, 0,
47. 58, 50, 42, 34, 26, 18, 10, 2,
48. 60, 52, 44, 36, 28, 20, 12, 4,
49. 62, 54, 46, 38, 30, 22, 14, 6
50. };
52. /* There are few lines of code missing */
53. }
55. int main (void)
56. {
57. unsigned char str[BLOCK_SIZE + 1] = "DummyStr";
58. block permutated = { 0x7a, 0xd2, 0x21, 0x3c, 0x05, 0x85, 0x8d, 0x71 };
60. permutateBlockRev(permutated);
61. strcpy(str, permutated);
62. printf("Password is: %s\n", str);
63. }

How to convert multi-page HTML e-book for Kindle

Recently I've got link to the interesting e-book Architecture of Open Source applications. Because I prefer to read books on my Kindle and there was no MOBI version I've decided to prepare it myself. Here is step-by-step guide on how to convert multi-page HTML to format suitable for Kindle on your Ubuntu/Debian/Other linux.

Getting data

First of all we need to get HTML/CSS and image files to local machine. On my machine it's as simple as:

$mkdir ~/aosabook; cd ~/aosabook
$wget -I en,images -nd -r -k -l 3 http://www.aosabook.org/en

We want to download all documents recursively but only from en and images directories, don't create directory structure to local copy and replace paths in html documents so they're locally referenced. For more details check man wget.

Convert to single HTML

Now we have all data downloaded in ~/aosabook and to check whether book is readable we just have to open file:///home/aosabook/ in browser.
Because the structure of the web is multi-page, we have to do additional step. Convert the multi page document into single page. I've used htmldoc utility for this. Run htmldoc and do following:

• input tab
  • choose Document Type: Book
  • Use add files button to add all html files from ~/aosabook/
  • Select cover image
• output tab
  • set output to file
  • set output path to ~/aosabook/aosabook.html
  • set output format to html

Play with some other options namely width (set it to 600px for kindle 3) and hit generate.

Convert to Mobi

In order to convert to MOBI format suitable for Kindle, I've just added generated aosabook.html as book to Calibre (you use calibre for your kindle management right?) and clicked on book to convert to Mobi and upload to device.