In the menu, under Help.
But don't worry looking there - I just checked it myself, and the MV documentation on the windowskin is dismally lacking.
If you happen to have Ace, look under help (same thing - menu > Help > Contents or just press F1) and go through RPG Maker VX Ace > Reference Material > Resource Standards > Window Skins.
Window skins are 128 × 128 images. You should normally use a 32-bit PNG file.
A
Window background 1. The 64 × 64 pattern is drawn by growing or shrinking to fit the actual window. Strictly speaking, it is two pixels smaller around the window. This is done to show a natural-looking window with rounded corners.
B
Window background 2. The 64 × 64 pattern is drawn tile style so as to cover background 1.
C
Window frames and arrows. Four corners that are 16 × 16 are drawn and the remaining frame (the sides) are drawn tile style at a 16-pixel thickness to match the window. Arrows are used as marks when scrolling window content.
D
The command cursor used to show items that are selected in windows. The two pixels on the periphery are expanded/contracted horizontally and vertically, and the rest is expanded/contracted evenly to fit the size of the cursor.
E
The pause sign used to show the waiting-for-button-press state in message windows. It animates based on four 16 × 16 patterns.
F
Text colors that can be used by control characters of the Show Text event command. You can place four rows of eight 8 × 8 images (colors) each.
A is the top left quarter
B is the bottom left quarter
C is the top right quarter
Now divide the remaining bottom right quarter into quarters itself. Of that group,
D is the top left quarter
E is the top right quarter
F is the whole bottom half
In Ace, A was stretched to cover the background, and B was tiled. I'm not sure if MV is the same, or if B is also stretched.