I am using the skyrim HD DLC and was wondering how is other texture packs working with this.

Cause i see it creates no texture folder so i guess it unpacks them on the fly and use them.
What hapening to the textures i put in the texture folders, if they are one of the same that uses in the HD DLC, will the custom textures always overwrite the
DLC , or will the dlc overwrite the custom textures ?

It uses a BSA and the files are packed in there. This keeps them close together on the HDD.

But will my other files i putted in texture folder work over the DLC ?

Yes, that was fixed in 1.4.x.