All vegan marshmallows - which contain no animal products and as such are parve - are kosher so long as they are kosher certified. The majority of kosher marshmallows on the market are made from fish...
Any brand of pudding that has a small OU symbol somewhere on the packaging is kosher. There are other kosher symbols you can look for as well. To see what they look like, try Googling: kosher symbol.