Place potato chunks in a large pot and cover with cold water by about an inch. Add eggs to the pot with the potatoes. Bring to a boil over high heat, then reduce to a simmer and cook for 12-15 minutes until potatoes are tender when pierced with a fork.
Carefully remove eggs with a slotted spoon and transfer to a bowl of ice water to cool. Drain potatoes and spread them on a baking sheet to cool completely—this helps prevent a soggy salad.
While potatoes cool, peel the eggs and chop them into small pieces. In a small bowl, whisk together mayonnaise, Dijon mustard, apple cider vinegar, sugar, salt, and pepper until smooth.
In a large mixing bowl, combine cooled potatoes, chopped eggs, celery, red onion, and dill. Pour the dressing over the top.
Gently fold everything together until evenly coated, being careful not to mash the potatoes. Taste and adjust seasoning if needed.
Cover and refrigerate for at least 30 minutes before serving to allow flavors to blend. Stir gently once more before transferring to a serving dish.