Remove Duplicates in Text File

So I have a list of names, but the names are duplicated down the page. Each name is on a new line by itself.

If I want to remove all the duplicates and just find out who is included in the file, I can simply do:

awk '!seen[$0]++' filename

This will remove duplicates and print them to your terminal.

WebStorm Encoded my HTML File as UTF-16BE

Well here’s the fix…

You have to replace the file. First get a UTF-8 output:

iconv -f UTF-16 -t UTF-8 [path/to/file]

Then delete the file. Make a new file (already UTF-8 by default) and paste in the UTF-8 output from your terminal.


Simple Shader with ThreeJS

Ok folks, it doesn’t get easier than this. If you want to start writing shaders in the browser but can’t figure out how to create your shader program, ThreeJS has you covered:

In your JS

var camera = new THREE.Camera();
	camera.position.z = 1;

	var scene = new THREE.Scene();

	var uniforms = {
		time: { type: "f", value: 1.0 },
		resolution: { type: "v2", value: new THREE.Vector2() }

	var material = new THREE.ShaderMaterial( {
		uniforms: uniforms,
		vertexShader: document.getElementById( 'vertexShader' ).innerText,
		fragmentShader: document.getElementById( 'fragmentShader' ).innerText

	var mesh = new THREE.Mesh( new THREE.PlaneGeometry( 2, 2 ), material );

	scene.add( mesh );

	var canvas = document.querySelector(".lense-flare");
	var parent = canvas.parentElement;
	var renderer = new THREE.WebGLRenderer({ canvas: canvas });
	renderer.setSize( parent.clientWidth, parent.clientHeight );

	uniforms.resolution.value.x = parent.clientWidth;
	uniforms.resolution.value.y = parent.clientHeight;
	var startTime =;


	function animate() {
		requestAnimationFrame( animate );

	function render() {
		var elapsedMilliseconds = - startTime;
		var elapsedSeconds = elapsedMilliseconds / 1000.;
		uniforms.time.value = 60. * elapsedSeconds;
		renderer.render( scene, camera );


In your head:

<script id="vertexShader" type="x-shader/x-vertex">
			uniform float time;
			uniform vec2 resolution;
			void main()	{
				gl_Position = vec4( position, 1.0 );
		<script id="fragmentShader" type="x-shader/x-fragment">
			uniform float time;
			uniform vec2 resolution;
			void main()	{
				float x = mod(time + gl_FragCoord.x, 20.) < 10. ? 1. : 0.;
				float y = mod(time + gl_FragCoord.y, 20.) < 10. ? 1. : 0.;
				gl_FragColor = vec4(vec3(min(x, y)), 1.);




Finding A Pixel in Canvas Image Data

I never have time, so this will be quick.

Let’s say we want to find a pixel index amoung all that image data canvas returns with:

var imgData = ctx.getImageData(x,y,w,h);

The formula is simple:

var y = [ROW];
var x = [COLUMN];
var w = imgData.width;
var pixIdx = (y * w + x) * 4;
var pixel = imgData[pixIdx];

Basically, you need to draw this one out. Let’s say we have a 3×3 image. Each pixel would have it’s own zero based index. The width would be three. Each row would stretch from 0 – 2 and each column would stretch from 0 – 2.



Now let’s say we want to select the 7th pixel. We know the formula is:

index => (y * w + x)

So we can calculate the index by selecting Row 2 and Column 1. Just like battleship.



The only confusing part is why we should multiply by 4 at the end! It’s simple really, there are 4 values for every pixel (r,g,b,a).

To loop through and get the index for every pixel, might look something like:

for(var x = 0, w = imgData.width; x < w; ++x) {
	for(var y = 0, h = imgData.height; y < h; ++y) {
		var pixIndex = (y * w + x) * 4;
		var r = imgData[pixIndex];
		var g = imgData[pixIndex + 1];
		var b = imgData[pixIndex + 2];
		var a = imgData[pixIndex + 3];

That’s it.

Mars VR Demo

Just wanted to post this. The source is pretty self explanatory and i’ll put up a github later once I clean up a few things.

I’ll explain later. Be patient, it takes a second to load.