You may only remove sand and loose soil on the putting green and not on the fringe or general area of the course.
If you are off the green and want to putt and there is sand on your line on the green and on the fringe you may wipe or brush the sand away on the green but may not touch any sand that is lying on the fringe (unless it arrived there after your ball had come to rest, perhaps via another players bunker shot).
Breaking this rule is a general penalty of two strokes in stroke play or loss of hole in match play under Rule 8.1a for improving the conditions affecting the stroke (line of play).