This is logical. When you create a group, you automatically become its first administrator. As an admin, you have the rights to decide not only about the group's name and avatar, but also who is allowed to participate and what's being posted in your group. These powers are assigned to you regardless if the group is public or private.
It goes without saying that all groups must conform to the House Rules, meaning that system administrators such as myself will also have admin access to your group.
All your administrator functions are grouped under an "Admin" tab in your group, which becomes available to you as you create it.
As a group administrator, it may be wise to share your administrator and/or moderator rights with other people. You can do this by selecting Admin --> Members. There you can promote selected group members to group administrator or moderator status.