DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Applicant is advised that should claim 1 be found allowable, claim 16 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 7, 10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 7 recites “from the first ratio to the second ratio, in response to an action by at least one of the plurality of persons, included in the first image, moving in the height direction.” however Examiner did not find support for determination of vertical action in the Specification. For example, Specification Paragraph 45 is directed “the panoramic image 203 in the case where some of the participants 1 20 are standing. The first image generation unit 62 increases the height of the 2 0 panoramic image 203 such that the panoramic image 203 includes faces of all the participants 120.” However, this is directed to detecting contents of a static image not to determining motion.
Claim 10 recites “wherein the first image includes an electronic device connected to the information processing apparatus via a network,” however Examiner did not find support for an image of a network connected device in the Specification.
Claim Construction
Note that, for purposes of compact prosecution, multiple reasons for rejection may be provided for a claim or a part of the claim. The rejection reasons are cumulative, and Applicant should review all the stated reasons as guides to improving the claim language and advancing the prosecution toward an allowance.
Claim scope is not limited by claim language that suggests or makes optional but does not require steps to be performed by a method claim, or by claim language that does not limit an apparatus claim to a particular structure. However, examples of claim language, although not exhaustive, that may raise a question as to the limiting effect of the language in a claim are: (A) “adapted to” or “adapted for” clauses; (B) “wherein” clauses; and (C) “whereby” clauses. M.P.E.P. 2111.04. Other examples are where the claim passively indicates that a function is performed or a structure is used without requiring that the function or structure is a limitation on the claim itself. The clause may be given some weight to the extent it provides "meaning and purpose” to the claimed invention but not when “it simply expresses the intended result” of the invention. In Hoffer v. Microsoft Corp., 405 F.3d 1326, 1329, 74 USPQ2d 1481, 1483 (Fed. Cir. 2005). Further, during prosecution, claim language that may or may not be limiting should be considered non-limiting under the standard of the broadest reasonable interpretation. See M.P.E.P. 904.01(a); In re Morris, 127 F.3d 1048, 44 USPQ2d 1023 (Fed. Cir. 1997).
Component arrangements or rearrangements which do not modify operation of the device cannot be relied upon to patentably distinguish the claimed invention from the prior art. In re Seid, 161 F.2d 229, 73 USPQ 431 (CCPA 1947); In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950) (shifting the position of the starting switch was not patentable because it would not have modified the operation of the device.
"[E]ven though product-by-process claims are limited by and defined by the process, determination of patentability is based on the product itself. The patentability of a product does not depend on its method of production. If the product in the product-by-process claim is the same as or obvious from a product of the prior art, the claim is unpatentable even though the prior product was made by a different process." In re Thorpe, 777 F.2d 695, 698, 227 USPQ 964, 966 (Fed. Cir. 1985). See MPEP 2113(I).
Claim 1-14 and 16 recite “circuitry; and one or more memories storing instructions that cause the circuitry to execute …” a generic term (circuitry) modified by functional language but not modified by structure or a structural term and not naming a structure readily recognized by persons of skill in the art to perform the claimed function. The limitation invokes 35 U.S.C. 112(f) or 35 U.S.C. 112 (pre-AIA ), sixth paragraph, and shall be construed to cover the corresponding structure described in the specification and equivalents thereof. Specification defines “The term "processing circuit or circuitry" used herein refers to a processor that is programmed to carry out each function by software,” on Page 56, lines 19-20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 7-9, 11-16 are rejected under 35 U.S.C. 103 as being unpatentable over US 9936162 to Gadnir (“Gadnir”) in view of US 20220070385 to Van (“Van”).
Regarding Claim 1: “An information processing apparatus comprising:
circuitry; and a memory storing instructions that cause the circuitry to execute: (“The memory 228 can be any computer readable medium, such as a random access memory (RAM) or other dynamic storage device ( e.g. dynamic RAM, static RAM, synchronous RAM, etc.) coupled to the bus 244 for storing information and instructions to be executed by the processor 236.” Gadnir, Column 8, lines 57-61.)
generating a first image to be displayed in a first display area; … generating a plurality of second images, [clipped from the first image], to be displayed in a plurality of second display areas, respectively; … generating a combined image in which the first image is arranged adjacent to the second images in a height direction; and (“The layout 500 includes first, second, and third windows [display areas] 504, 508, and 512, respectively. Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration and any number of windows” Gadnir, Column 12, lines 16-21 and arranged in a height direction in Fig. 5. Note that the prior art examples are similar to Figs 2A-2C in the Specification.)
a plurality of second images, clipped from the first image, (“wide angle digital image information can permit extraction of multiple high quality images, which can be displayed in the windows as desired.” Gadnir, Column 12, lines 22-35. See similarly in Therkelsen, Paragraph 26.)
changing a ratio of a dimension of the first display area in the height direction relative to dimensions of the second display areas in the height direction from a first ratio to a second ratio different from the first ratio.” (Under the broadest reasonable interpretation consistent with the specification and ordinary skill in the art, c. changing a ratio of a dimension of the first display area in the height direction can be directed to zooming, cropping, or resizing the image to properly fit the targeted objects on the screen.
Prior art teaches this functionality. First, prior art selects optimum sizes of the images, including the claimed first image as “the imaging controller 256 selects as the optimal view a view having the first, second, and third participants 300a-c in frame, in focus and centralized.” Gadnir, Column 10, lines 58-63. Also, for the second area including third and fourth images of the individuals, prior art “acquires and digitally crops and/or zooms an image of the selected object from the digital image and optionally normalizes or resizes the selected object image so that all of the images for the different objects appear to be equally sized … The normalization may be accomplished by resizing the width and/or height of the facial image … vertical scaling may be performed … Other scaling techniques may alternatively be used.” Gadnir, Column 14, lines 30-48 and example layout in Fig. 5. Finally, prior art teaches sizing and arranging the windows to shit the images that are displayed. the images are displayed, where “Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration …” Gadnir, Column 12, lines 17-23.
Thus, both the image shapes and sizes (heights, widths) and the image layout configuration on display can be adjusted to make sure the optimal viewing of the participants is arranged on the screen. See additional treatment of window resizing below.)
To the extent the claim is directed to resizing the window which contains an image on the screen, Gadnir does not teach that resizing the image to optimize display of the content is connecting to resizing the window in which the image is displayed. As noted above, Gadnir indicates that any configuration can be selected for the windows and for the images.
Van explicitly teaches resizing the window with the image in the context of displaying images of people and objects: “shifts and resizes the position of various participant windows 1255, as shown in FIG. 12AP. In some embodiments, device 600 enlarges a respective participant window when the participant is speaking. For example, in FIG. 12AP, device 600 detects audio 1276 (e.g., laughter) from the woman represented in participant image data 1204-2 and, in response, enlarges the woman's participant window 1255-2.” Van, Paragraph 476.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of Gadnir to resize the image as well as the window in which the image is displayed as taught in Van, “for displaying visual effects in a live video communication session”. Van, Paragraph 477.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness. Adjusting images and application windows for optimal view or for visual effect are well established features in the field of personal computers.
Regarding Claim 2: “The information processing apparatus according to claim 1, wherein the first image is a panoramic image generated from a wide-angle image captured by an image-capturing device.” (First note that the content or format of the first image or description of the first image by process of making it does not limit the claimed apparatus or method to performing a particular step. Cumulatively, Prior Art teaches using such image format: “the high resolution and wide angle digital image information can permit extraction of multiple high quality images, which can be displayed in the windows as desired.” Gadnir, Column 12, lines 33-55. In this case a panoramic image can be a whole or a part of a wide angle image from a camera that “has a horizontal field or angle of view of at least about 100 degrees, more typically at least about 110 degrees, and more typically at least about 120 degrees.” Gadnir, Column 11, lines 46-50.)
Regarding Claim 3: “The information processing apparatus according to claim 1, wherein the first image includes a plurality of persons.” .” (First note that the content or format of the first image or description of the first image by process of making it does not limit the claimed apparatus or method to performing a particular step. Cumulatively, Prior Art teaches using images including persons: “The participant monitor 252 can acquire the facial images of each participant in the captured image using face detection techniques, acquire other object images in the captured image (such as a whiteboard, table, chair, and the like) using digital processing techniques, determine an identity of each acquired facial image by face recognition techniques using an identified biometric information of the participant,” Gadnir, Column 9, lines 37-39.)
Regarding Claim 4: “The information processing apparatus according to claim 2, wherein the first image includes a plurality of persons.” (This claim is rejected for reasons stated for Claim 3 in view of the Claim 2 rejection.)
Regarding Claim 7: “The information processing apparatus according to claim 3, wherein the circuitry is caused to change the ratio of the dimension of the first display area in the height direction relative to the dimensions of the second display areas in the height direction from the first ratio to the second ratio, … in response to an action by at least one of the plurality of persons, included in the first image, moving in the height direction.” (See rejection under section 112 above. Cumulatively, prior art teaches an example of this: “the imaging controller 256 selects as the optimal view a view having the first, second, and third participants 300a-c in frame, in focus and centralized, with minimal background in the captured image. The imaging controller 256 adjusts the pan, tilt, and zoom of the camera 216 to produce this view. After the video conferencing communication session commences and after a selected time interval has elapsed, the second participant 300b becomes the active speaker and stands and walks to the whiteboard 304. The imaging controller 256, in response, selects as the optimal view a view having the whiteboard and second participant in frame, in focus and centralized,” including height and pan adjustments. Gadnir, Column 12, lines 48-53. See adjusting the window in correspondence with the image in Claim 1.)
Regarding Claim 8: “The information processing apparatus according to claim 1, wherein the circuitry is caused to display the first image above the second images in the height direction.” (“The layout 500 includes first, second, and third windows [display areas] 504, 508, and 512, respectively. Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration and any number of windows” thus either of the vertical arrangement of first and second images is obvious. Gadnir, Column 12, lines 16-21 and Fig. 5. Note that the prior art examples are similar to Figs 2A-2C in the Specification.)
Regarding Claim 9: “The information processing apparatus according to claim 1, wherein the dimension of the first image in the height direction in a case of the second ratio is greater than the dimension of the first image in the height direction in a case of the first ratio.” (“Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration …” Gadnir, Column 12, lines 17-23. Also note that “shifts and resizes the position of various participant windows 1255, as shown in FIG. 12AP. In some embodiments, device 600 enlarges a respective participant window when the participant is speaking. For example, in FIG. 12AP, device 600 detects audio 1276 (e.g., laughter) from the woman represented in participant image data 1204-2 and, in response, enlarges the woman's participant window 1255-2.” Van, Paragraph 476. Thus enlarging one window and reducing another is an obvious application in the prior art. See statement of motivation in Claim 1.)
Regarding Claim 11: “The information processing apparatus according to claim 1, wherein for the plurality of second images are arranged horizontally next to each other.” (“The layout 500 includes first, second, and third windows [display areas] 504, 508, and 512, respectively. Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration and any number of windows” thus a horizontal arrangement of second images is obvious. Gadnir, Column 12, lines 16-21 and Fig. 5. Note that the prior art examples are similar to Figs 2A-2C in the Specification.)
Regarding Claim 12: “The information processing apparatus according to claim 1, wherein the circuitry is caused to change the ratio of the dimension of the first display area in the height direction relative to the dimensions of the second display areas in the height direction from the first ratio to the second ratio, … while maintaining a dimension of the first display area and a total dimension of the second display areas in a horizontal direction perpendicular to the height direction.” (“The layout 500 includes first, second, and third windows [display areas] 504, 508, and 512, respectively. Each of the first, second, and third windows 504, 508, and 512 displays a corresponding image. As will be appreciated, the layout 500 can have any configuration and any number of windows … The normalization may be accomplished by resizing the width and/or height of the facial image,” thus either of the vertical arrangement of first and second images is obvious. Gadnir, Column 12, lines 16-21, Column 14, lines 37-40, and Fig. 5. See treatment of resizing windows to accommodate images in Claim 1. Note that the prior art examples are similar to Figs 2A-2C in the Specification.)
Regarding Claim 13: “The information processing apparatus according to claim 1, further comprising a display, wherein the circuitry is caused to display the combined image on the display.” (“The exemplary endpoint 108 comprises a display device 200, a controller 204, a capture device 208, and a control unit 212. … The images positioned in each window can be based on a number of criteria, such as current active speaker participant … wide angle digital image information can permit extraction of multiple high quality images, which can be displayed in the windows as desired.” ,” Gadnir, Column 6, lines 41-44, Column 12, lines 22-35.)
Regarding Claim 14: “The information processing apparatus according to claim 1, wherein the circuitry is caused to transmit the combined image to a communication terminal connected with the information processing apparatus via a network.” (“The network video conference unit 104 can service any conference topology … any suitable devices for providing a user interface for a voice or video conference. Some of the endpoints [terminas] can be capable of hosting the voice portion of the conference only or a part of the video conference … or all of the video conference ( e.g., display images of remote participants and transmit an image of the local participant).” Gadnir, Column 6, lines 9-24.)
Claim 15, “A display method,” is rejected for reasons stated for Claim 1, because the apparatus of Claim 1 performs the method of Claim 15.
Claim 16 “An information processing system,” is rejected for reasons stated for Claim 1.
Claims 5-6, 10 are rejected under 35 U.S.C. 103 as being unpatentable over US 9936162 to Gadnir (“Gadnir”) in view of US 20220070385 to Van (“Van”) in view of US 20200351435 to Therkelsen (“Therkelsen “).
Regarding Claim 5: “The information processing apparatus according to claim 1, wherein the plurality of second images are generated based on detected sound information.” (First note that the content or format of the first image or description of the first image by process of making it does not limit the claimed apparatus or method to performing a particular step.
Cumulatively, Gadnir and Van do not teach the above claim feature.
Therkelsen teaches this claim feature in the context of capturing conference video: “alter one of the video outputs of the video cameras 112(1)-112(3) to present a close-up or zoomed in view of the actively speaking participant (e.g., 106(2) or 106(3) as illustrated in FIG. 2).” Therkelsen, Paragraph 26. Also note that a close up image of the participant includes an image of the detected microphone as noted in Therkelsen, Paragraph 42 and Fig. 8C.)
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of Gadnir and Van to generate second images based on detected sound information as taught in Therkelsen, in order “to present a close-up or zoomed in view of the actively speaking participant.” See, Therkelsen, Paragraph 26 and Fig. 2.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Regarding Claim 6: “The information processing apparatus according to claim 3, wherein at least one of the plurality of second images includes a person speaking among the plurality of persons.” (First note that the content or format of the first image or description of the first image by process of making it does not limit the claimed apparatus or method to performing a particular step. Cumulatively, Prior Art teaches “alter one of the video outputs of the video cameras 112(1)-112(3) to present a close-up or zoomed in view of the actively speaking participant (e.g., 106(2) or 106(3) as illustrated in FIG. 2).” Therkelsen, Paragraph 26. Also note that a close up image of the participant includes an image of the detected microphone as noted in Therkelsen, Paragraph 42 and Fig. 8C. See statement of motivation in Claim 5.)
Regarding Claim 10: “The information processing apparatus according to claim 1, wherein the first image includes an electronic device connected to the information processing apparatus via a network.” (“acquire other object images in the captured image (such as a whiteboard, table, chair, and the like)” Gadnir, Column 9, lines 37-39. The object can be an electronic devices, such as a microphone “detecting/recognizing the microphones 118 (1)-118(2) in the video outputs of the video cameras 112 … one or more microphones 118, encodes the captured audio/video into data packets, and transmit the data packets to other endpoints 104(2)-104(N) or to the conference server.” See Therkelsen, Paragraphs 25, 19, Fig. 2, and statement of motivation in Claim 5.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20160253859 Staffer (“Staffer”) as cited in previous office actions and relevant for recognizing bar codes in the images.
US 20220245850 to Liu (“Liu”) as cited in previous office actions and relevant for using fisheye images for generating panoramic images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIKHAIL ITSKOVICH whose telephone number is (571)270-7940. The examiner can normally be reached Mon. - Thu. 9am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at (571)272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIKHAIL ITSKOVICH/Primary Examiner, Art Unit 2483
Prosecution Insights