Season the chicken with ½ teaspoon of the salt and ¼ teaspoon of the pepper. Heat the oil in a large heavy skillet on medium-high. Add the chicken and sear it until browned all over, 4 to 6 minutes per side, turning once (cook in batches or use two pans if necessary). Transfer the thighs to a 4-quart slow cooker.
Add the onion, carrots, and celery to the skillet and cook over medium heat until soft, 4 minutes. Add the mushrooms and garlic and cook until the mushrooms begin to release their liquid, 4 minutes. Add the wine and simmer over high heat until almost all the liquid evaporates, 5 minutes.
Transfer the pan contents to the slow cooker. Stir in the broth, barley, and rosemary. Cover and cook the soup on low for 4 hours or high for 2 hours.
Use tongs to transfer the chicken to a cutting board; shred the chicken, discarding the bones. Stir the shredded chicken into the soup along with the dill, lemon juice, and remaining ½ teaspoon of salt and ¼ teaspoon of pepper.
Tip: To make this dish on the stove top, swap a large soup pot for the large skillet and slow cooker. When all the liquid is in the pot, cover and simmer the soup gently over low heat until the chicken is tender enough to shred, about 1½ to 2 hours.