Place potato cubes in a large pot and cover with cold water by 1 inch. Bring to a boil over high heat.
Reduce heat to medium and simmer for 10-12 minutes, until potatoes are tender when pierced with a fork but not falling apart.
Drain potatoes in a colander and spread them on a baking sheet to cool completely, about 30 minutes.
While potatoes cool, make the dressing: In a medium bowl, whisk together mayonnaise, mustard, vinegar, salt, and pepper until smooth.
In a large mixing bowl, combine cooled potatoes, celery, onion, chopped eggs (reserving one egg for garnish), and pickle relish.
Pour dressing over potato mixture and gently fold everything together until evenly coated.
Cover bowl with plastic wrap and refrigerate for at least 1 hour to allow flavors to meld.
Before serving, slice reserved hard-boiled egg and arrange on top. Sprinkle with paprika for color.
