Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant amendments to claim 20 have overcome 35 USC 101 rejection.
Applicant's arguments filed on 04/28/2026 have been fully considered but they are not persuasive.
Applicant argues regarding claims 1, 19 and 20 that Kajimura fails to teach “determine a change in the respective zoom magnification corresponding to the each frame of the plurality of frames, based on the switch to the second lens from the first lens; and control the simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification”. Applicant states that the reason for this is that Kajimura merely describes displaying an image with a plurality of frames having different field angles corresponding to different lenses. However, Kajimura does not describe determining a change in a respective zoom magnification of each frame of the plurality of frames, based on switching to a second lens from a first lens. Accordingly, Kajimura does not describe simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification (On Pages 13 and 14 of Response). The Examiner respectfully disagrees.
Examiner’s response:
First an explanation regarding the basics of the focal length and magnification. Lens focal length tells us the angle of view—how much of the scene will be captured—and the magnification—how large individual elements will be. The longer the focal length, the narrower the angle of view and the higher the magnification. The shorter the focal length, the wider the angle of view and the lower the magnification.
Kajimura teaches that FIG. 3A shows a state where an image taken with the A lens having the shortest focal length is displayed on the screen of the display unit 14. The field angle range of the image is represented as field angle frames 40a, 40b, 40c, and 40d. In FIG. 3A, for the convenience of explanation, the display frames of the angle-of-view frames 40b, 40c, and 40d of each lens other than the A lens are also displayed in the display screen. Such a display image is an example, and for example, only a display frame of the view angle frame 40d may be displayed, or a display frame of a part of the view angle frame may be displayed. When shooting is performed in the state of FIG. 3A, images obtained by the image sensor 6 through the lenses having different focal lengths are generated by the image processing unit 12 as shown in FIGS. 3B to 3E. The images shown in FIGS. 3B to 3E are images obtained by the respective imaging elements 6 through the respective imaging optical systems. Therefore, unlike the case of performing enlargement processing such as electronic zoom, the resolution of each image is the same, so that a captured image at a long focal length can be obtained with high definition.
Therefore 40a corresponds to shortest focal length lens and the widest angle of view and the lowest magnification. 40b corresponds to the next larger focal length lens and narrower angle of view than 40a and higher magnification than 40a. 40c corresponds to the next larger focal length lens than 40a and 40b and higher magnification than 40a and 40b. 40d is the longest focal length and the narrowest angle of view and the highest magnification.
Kajimura teaches that they are doing this because unlike the case of performing enlargement processing such as electronic zoom, the resolution of each image is the same, so that a captured image at a long focal length can be obtained with high definition.
Therefore each image is taken with different zoom magnification with simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification based on a switch to different lens and the claim limitation “determine a change in the respective zoom magnification corresponding to the each frame of the plurality of frames, based on the switch to the second lens from the first lens; and control the simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification” is therefore met.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6-13, 15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Iijima et al. (US PGPUB 20100277620) in view of Kajimura (JP Patent # 2015005891).
[Claim 1]
Iijima teaches an information processing device, comprising a central processing unit (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) configured to control display of a through image on a display screen (e.g. fig. 4), control simultaneous simultaneously display of a plurality of frames of a user interface on the display screen, wherein each frame of the plurality of frames corresponds to a respective zoom magnification of a plurality of zoom magnifications of a camera which a through image is displayed (Paragraph 72, fig. 4, The view angle candidate frame generation unit 121a generates the view angle candidate frames corresponding to the set candidate values. The view angle candidate frame display unit 122 superimposes the view angle candidate frames generated by the view angle candidate frame generation unit 121a on the input image so as to generate the output image. An example of the output image generated in this way is illustrated in the middle part of FIG. 4. An output image PA2 illustrated in the middle part of FIG. 4 is obtained by superimposing a view angle candidate frame FA1 corresponding to the candidate value of times. 4, a view angle candidate frame FA2 corresponding to the candidate value of times. 8, and a view angle candidate frame FA3 corresponding to the candidate value (upper limit value) of times. 12 on the input image under the current zoom magnification of times. 1.).
Iijima fails to teach the camera includes a plurality of lenses, control the camera to switch from a first lens of the plurality of lenses to a second lens of the plurality of lenses; determine a change in the respective zoom magnification corresponding to the each frame of the plurality of frames, based on the switch to the second lens from the first lens; and control the simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification.
However Kajimura teaches that FIG. 3A shows a state where an image taken with the A lens having the shortest focal length is displayed on the screen of the display unit 14. The field angle range of the image is represented as field angle frames 40a, 40b, 40c, and 40d. In FIG. 3A, for the convenience of explanation, the display frames of the angle-of-view frames 40b, 40c, and 40d of each lens other than the A lens are also displayed in the display screen. Such a display image is an example, and for example, only a display frame of the view angle frame 40d may be displayed, or a display frame of a part of the view angle frame may be displayed. When shooting is performed in the state of FIG. 3A, images obtained by the image sensor 6 through the lenses having different focal lengths are generated by the image processing unit 12 as shown in FIGS. 3B to 3E. The images shown in FIGS. 3B to 3E are images obtained by the respective imaging elements 6 through the respective imaging optical systems. Therefore, unlike the case of performing enlargement processing such as electronic zoom, the resolution of each image is the same, so that a captured image at a long focal length can be obtained with high definition.(Paragraph 17). Therefore 40a corresponds to shortest focal length lens and the widest angle of view and the lowest magnification. 40b corresponds to the next larger focal length lens and narrower angle of view than 40a and higher magnification than 40a. 40c corresponds to the next larger focal length lens than 40a and 40b and higher magnification than 40a and 40b. 40d is the longest focal length and the narrowest angle of view and the highest magnification.
[an explanation regarding the basics of the focal length and magnification. Lens focal length tells us the angle of view—how much of the scene will be captured—and the magnification—how large individual elements will be. The longer the focal length, the narrower the angle of view and the higher the magnification. The shorter the focal length, the wider the angle of view and the lower the magnification.]
Therefore taking the combined teachings of Iijima and Kajimura, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have the camera includes a plurality of lenses, control the camera to switch from a first lens of the plurality of lenses to a second lens of the plurality of lenses; determine a change in the respective zoom magnification corresponding to the each frame of the plurality of frames, based on the switch to the second lens from the first lens; and control the simultaneous display of the plurality of frames based on the determination of the change in the respective zoom magnification in order to have the same resolution for all zoomed images and visually grasp how a framed image is acquired with a lens having different focal length.
[Claim 2]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to select a frame from the plurality of frames based on a user input, and zoom the through image at the respective zoom magnification corresponding to the selected frame (Paragraph 77, if the user determines one of the view angle candidate frames (YES in STEP 4), the zoom in operation is performed so that the image having the angle of view of the determined view angle candidate frame is obtained (STEP 5), and the operation is finished. In other words, the zoom magnification is changed to the candidate value corresponding to the determined view angle candidate frame, and the operation is finished. If the view angle candidate frame FA3 is determined in the output image PA2 illustrated in the middle part of FIG. 4, for example, an output image PA3 illustrated in the lower part of FIG. 4 having substantially the same angle of view as the view angle candidate frame FA3 is obtained by the zoom in operation).
[Claim 3]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to select a frame from the plurality of frames based on a user input, and control an imaging operation of the camera at the respective zoom magnification corresponding to the selected frame (Paragraph 77, if the user determines one of the view angle candidate frames (YES in STEP 4), the zoom in operation is performed so that the image having the angle of view of the determined view angle candidate frame is obtained (STEP 5), and the operation is finished. In other words, the zoom magnification is changed to the candidate value corresponding to the determined view angle candidate frame, and the operation is finished. If the view angle candidate frame FA3 is determined in the output image PA2 illustrated in the middle part of FIG. 4, for example, an output image PA3 illustrated in the lower part of FIG. 4 having substantially the same angle of view as the view angle candidate frame FA3 is obtained by the zoom in operation).
[Claim 4]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to determining the respective zoom magnification corresponding to the each frame of the plurality of frames on a current zoom magnification of the through image (Paragraph 71, In this case, candidate values of the changed zoom magnification are set. As a method of setting the candidate values of the zoom magnification, for example, values obtained by dividing equally between the currently set zoom magnification and the upper limit value of the zoom magnification, and the upper limit value may be set as the candidate values. Specifically, for example, when it is supposed that the currently set zoom magnification is .times.1, the upper limit value is .times.12, and values obtained by dividing equally into three are set as candidate values, .times.12, .times.8, and .times.4 are set as the candidate values).
[Claim 6]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to detect a subject in the through image and control display of the each frame of the plurality of frames based on the detected subject (Paragraph 86, As illustrated in FIG. 5, a display image processing unit 12b of this example includes a view angle candidate frame generation unit 121b which generates the view angle candidate frames based on the zoom information and the object information, and outputs the same as view angle candidate frame information, and a view angle candidate frame display unit 122).
[Claim 7]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to determine a display position of the each frame of the plurality of frames on a position of the detected subject (Paragraph 92, In this example, the view angle candidate frame generation unit 121b generates the view angle candidate frames so as to include the object in the input image (STEP 2b). Specifically, if the object is a human face, the view angle candidate frames are generated as a region including the face, a region including the face and the body, and a region including the face and the peripheral region. In this case, it is possible to determine the zoom magnifications corresponding to the individual view angle candidate frames from sizes of the view angle candidate frames and the current zoom magnification).
[Claim 8]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to determine the respective zoom magnification corresponding to the each frame of the plurality of frames based on a size of the detected subject (Paragraph 87, The object information includes, for example, information about a position and a size of a human face in the input image detected from the input image, and information about a position and a size of a human face that is recognized to be a specific face in the input image and Paragraph 91, Thus, the view angle candidate frame generation unit 121b recognizes not only the currently set zoom magnification and the upper limit value but also a position and a size of the object in the input image).
[Claim 9]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to detect a plurality of subjects in the through image, wherein the plurality of
subjects includes the subject and control the simultaneous display of the plurality of frames corresponding to each of the detected plurality of the subjects (Paragraph 122, view angle candidate frames FB511 to FB513 are generated based on a plurality of objects D51 and D52 as illustrated in FIG. 12. For instance, view angle candidate frames FB511 to FB513 are generated based on barycentric positions of the plurality of objects D51 and D52. Specifically, for example, the view angle candidate frames FB511 to FB513 are generated so that barycentric positions of the plurality of objects D51 and D52 substantially match center positions of the view angle candidate frames FB511 to FB513).
[Claim 10]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to display the frame corresponding to the subject selected by a user out of a plurality of the detected subjects (Paragraph 141, fig. 14, Specifically, for example, when the user designates a position of an object D71 in an output image PB70 for which the view angle candidate frames are not generated, view angle candidate frames FB711 to FB713 are generated based on the object D71 as in an output image PB71).
[Claim 11]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to determine a size of the each frame of the plurality of frames based on a current zoom magnification of the through image (Paragraph 73, Therefore, based on the current zoom magnification and the candidate values, positions and sizes of the view angle candidate frames FA1 to FA3 can be set).
[Claim 12]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to emphasize a frame of the plurality of frames to recommended the frame to a user (Paragraph 75, fig. 4, For instance, the temporarily determined view angle candidate frame may be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thick line or a solid line while other view angle candidate frames that are not being temporarily determined may not be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thin line or a broken line).
[Claim 13]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to detect a plurality of subjects in the through image;
control the simultaneous display of the plurality of frames corresponding to a
first subject of the plurality of subjects; and
emphasize a first frame of the plurality of frames , wherein the first frame excludes a second subject of the plurality of subjects, the second subject corresponds to a second frame of the plurality of frames, and the second frame is different from the first frame (Paragraph 75, fig. 4, For instance, in the case where the operating unit 17 has a configuration including a zoom key (or cursor key) and an enter button, the user operates the zoom key so as to change a temporarily determined view angle candidate frame in turn, and presses the enter button so as to determine the temporarily determined view angle candidate frame. When the decision is performed in this way, it is preferred that the view angle candidate frame generation unit 121a display the view angle candidate frame FA3 that is temporarily determined by the zoom key in a different shape from others as illustrated in the middle part of FIG. 4 as the output image PA2, so that the temporarily determined view angle candidate frame FA3 may be discriminated. For instance, the temporarily determined view angle candidate frame may be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thick line or a solid line while other view angle candidate frames that are not being temporarily determined may not be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thin line or a broken line).
[Claim 15]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to emphasize the frame corresponding to the respective zoom magnification executable by an optical zooming process (Paragraph 75, fig. 4).
[Claim 17]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to display a button to select a frame of the plurality of the frames on the display screen and the frame is selected based on a user input on the displayed button (Paragraph 87, Note that, the object information is not limited to information about the human face, and may include information about a position and a size of a specific color part or a specific object (e.g., an animal), which is designated by the user via the operating unit 17 (a touch panel or the like) in the input image in which the designated object or the like is detected and Paragraph 75, fig. 4, Note that, in the case where the operating unit 17 is constituted of a touch panel or other unit that can specify any position, the view angle candidate frame that is closest to the position specified by the user may be determined or temporarily determined).
[Claim 18]
Iijima teaches wherein the CPU (CPU 15, fig. 1, Paragraph 49, the imaging device 1 includes a central processing unit (CPU) 15 which controls the entire operation of the imaging device 1) is further configured to select a frame of the plurality of frames based on a user input at any position in the frame (Paragraph 75, fig. 4, For instance, in the case where the operating unit 17 has a configuration including a zoom key (or cursor key) and an enter button, the user operates the zoom key so as to change a temporarily determined view angle candidate frame in turn, and presses the enter button so as to determine the temporarily determined view angle candidate frame. When the decision is performed in this way, it is preferred that the view angle candidate frame generation unit 121a display the view angle candidate frame FA3 that is temporarily determined by the zoom key in a different shape from others as illustrated in the middle part of FIG. 4 as the output image PA2, so that the temporarily determined view angle candidate frame FA3 may be discriminated. For instance, the temporarily determined view angle candidate frame may be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thick line or a solid line while other view angle candidate frames that are not being temporarily determined may not be emphasized by displaying the entire perimeter of the angle of view indicated by the relevant view angle candidate frame with a thin line or a broken line).
[Claims 19 and 20]
These are method and computer readable medium claims corresponding to apparatus claim 1 and are analyzed and rejected based upon apparatus claim 1.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Iijima et al. (US PGPUB 20100277620), Kajimura (JP Patent # 2015005891) and in further view of Watanabe et al. (US PGPUB 20120113307).
[Claim 14]
Iijima in view of Kajimura fails to teach detect a subject in the through image and emphasize, based on a speed of the subject detected in the through image is equal to a threshold or higher than the threshold, a largest frame out of the plurality of frames. However Watanabe teaches in FIG. 41A, FIG. 41B, and FIG. 41C show examples of a display image when the effect is changed in accordance with a movement speed of such an object of interest. For example, when a movement speed of an object of interest OI is lower than that in FIG. 41A, an area of a portion surrounded by an ellipse as an effect EF is reduced as shown in FIG. 41B. Alternatively, for example, when a movement speed of the object of interest OI is lower than that in FIG. 41A, a pattern of the ellipse as the effect EF is changed to vary an area of the effect, e.g., thin the line as shown in FIG. 41C and/or vary a degree of processing of the effect, e.g., lighten a color of the line. Moreover, both an area of a portion surrounded by the ellipse as the effect and a pattern of the line may be changed in accordance with a movement speed of the object of interest (Paragraph 244). Therefore taking the combined teachings of Iijima, Kajimura and Watanabe, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have emphasized, in a case where a speed of a subject detected in the through image is a threshold or higher, a larger frame out of the plurality of frames in order to emphasize the speed of the object based on the type of the frame.
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over Iijima et al. (US PGPUB 20100277620), Kajimura (JP Patent # 2015005891) and in further view of Anthony et al. (US PGPUB 20130243408).
[Claim 16]
Iijima in view of Kajimura fails to teach wherein a frame is displayed in a translucent state on the display. However Anthony teaches Translucent/disappearing control bar 910 may be used in some embodiments to maximize the display size of video display area 220 at a given aspect ratio. Translucent/disappearing control bar 910 may be configured to allow video information to show through the control bar 910 (Paragraph 68). Therefore taking the combined teachings of Iijima, Kajimura and Anthony, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have a frame displayed in a translucent state on the display in order to maximize the display size by allowing information to show through the translucent state.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESH K AGGARWAL whose telephone number is (571)272-7360. The examiner can normally be reached Monday - Friday 9:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 5712727564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESH K AGGARWAL/Primary Examiner, Art Unit 2637
Prosecution Insights