DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 2-3, 6, 8, 11, 13-14, and 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments, see applicant’s remarks, filed 11/19/2025, with respect to the rejection(s) of claim(s) 1, 4-5, 7, 9-10, 12, 15-17 and 20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Lee et al.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-5, 12, 15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Ohtsuka (US 5,847,870) in view of
Gardner, Brian, “The Dynamic Floating Window: a new creative tool for 3D movies”, Proceedings Volume 7863, Stereoscopic Displays and Applications XXII; 78631A (2011) and in further view of
Yang (US 2017/0295357 A1) and
Lee et al. (US 2004/0233222 A1).
Regarding claim 12, Ohtsuka discloses:
An system comprising: (Ohtsuka [1:10-16]: The present invention relates to binocular stereoscopic image display methods. More particularly, the present invention relates to a binocular stereoscopic image display method for displaying a stereoscopic image by projecting a plurality of actual images taken by a camera or the like or a plurality of images formed by computer graphics onto a lenticular lens or a polarization screen.)
determining a volume and depths associated with image content captured (Ohtsuka, [3:40-57]: right and left images taken by a stereoscopic camera, where image of an object is displayed if contained with the wedge-shaped space defined by a1, a-2, c1, c-2, c3, c4, - see Fig. 1, volume associated with image content; Fig. 5; [3:40-57] also discusses position of object relative to projection plane Ps – See Fig. 1 – i.e. depth);
defining, within the volume and based on the depths, a viewing range in which stereoscopic effects occur (Ohtsuka, Figs. 4A-4B and [4:42-60]: By employing right and left images I.sub.R and I.sub.L shown in FIG. 4B, a state where virtual frame VF perceptionally emerges in the space in front of actual projection plane P.sub.s is realized as shown in the bird's-eye view of FIG. 4A. The width (more strictly, the angle) of virtual frame VF must be as great as or greater than length c1 - c1 ' (or c4 '- c4) in FIG. 3 in order to create the correct occluding conditions geometrically);
generating, for rendering, modified image content with the resolved depth conflict, the modified image content including portions of the image content replaced by the at least one user interface element. (Ohtsuka, Figs. 4A-4B and [4:42-60] discloses display of virtual frame VF; [5:9-31]: an object located in front the projection plane can be perceived at the correct position and distortion in depth perception can be eliminated by calculating the corresponding points accurately to avoid vignetting due to the picture frame of the projection plane, or by making it appear as if an occluding object at the frontmost plane, i.e. the virtual frame, were present … the position of the virtual frame can be varied by the observer, or can be set automatically at a predetermined position in accordance with the presented image or set adaptively by calculating the content of the image)
Ohtsuka does not explicitly disclose the resolving the depth conflict using the user interface movement in response to detecting the image content extends beyond a boundary associated with the viewing range as claimed.
Gardner discloses:
determining a depth conflict between the image content and a boundary associated with the viewing range, the determining including detecting that at least one portion of the image content extends beyond the boundary associated with the viewing range (Gardner, p. 6, ¶1: We can use this Dynamic Floating Window (DFW) technique to remove the cause of window violations. By moving the virtual proscenium forward to match or be in front of the object that is causing the window violation, we eliminate the window violation, without increasing the size of the Retinal Rivalry Zones; Also p. 6, ¶3: “If there is a substantial window violation on only one side of the image, the virtual proscenium may be rotated to position that side of itself in front of the violating object.”; Figs. 5A-5B and Figs. 7A-7C);
resolving the depth conflict for the at least one portion using the viewing range and at least one user interface element (Gardner, p. 6, ¶1: We can use this Dynamic Floating Window (DFW) technique to remove the cause of window violations. By moving the virtual proscenium forward to match or be in front of the object that is causing the window violation, we eliminate the window violation, without increasing the size of the Retinal Rivalry Zones; Figs. 5A-5B: the virtual proscenium is rotated to a position in front of the object (on the Left side of the image), without increasing Retinal Rivalry Zone on the other (Right) side of the image; p. 6, ¶2: “If the object is moving, this virtual proscenium may be correspondingly moved, updated, or actively animated over a range of frames to continuously prevent the violation”); and
generating, for rendering, modified image content with the resolved depth conflict, the modified image content including portions of the image content replaced by the at least one user interface element (Gardner, p. 6, ¶1: remove window violations by moving virtual proscenium – see Figs. 5A-5B; also p. 7 and Figs. 7A-7B shows rotated proscenium removing lamp and portion of fire hydrant)
Both Ohtsuka and Gardner are directed to using virtual viewing frame elements to improve stereoscopic images based on edge violations. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, by utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, using known electronic interfacing and programming techniques. The modification results in an improved display of stereoscopic images for moving images by allowing for removal of dynamic window violations, diminishing eye fatigue, and maintaining good image quality (see p. 6 of Gardner discussing the dynamic floating window solving important problems).
Ohtsuka modified by Gardner do not explicitly disclose the particular computer architecture as claimed, including image content captured by a plurality of stereo cameras.
Yang discloses:
An image processing system including at least one processing device (Yang, Abstract, Figs. 1 and 3 and ¶40: communication device 10; ¶43 discusses system including communication device 10 in communication with any other compatible video communication device; ¶66: multiple users for 3D communication in video conference);
a plurality of stereo cameras (Yang, Abstract, Figs. 1 and 3 and ¶40: 3D communication device, where coupled to the controller 20 are two or more camera elements 30A-B, which work together to form a stereoscopic or 3D camera 40, which is configured to obtain or capture a stereoscopic or three-dimensional (3D) image; ¶62 and ¶66: multiple users with 3D communication device 10 to video conference with each other in 3D); and
a memory storing instructions that when executed cause the system to perform operations including: (Yang, ¶40: the communication device or apparatus 10 includes a controller/processor 20 that includes the necessary hardware, software or combination thereof for carrying out the functions to be discussed; Fig. 1 and ¶43: memory unit 70)
image content captured by the plurality of stereo cameras (Yang, Figs. 1 and 3 and ¶40: 3D communication device, where coupled to the controller 20 are two or more camera elements 30A-B, which work together to form a stereoscopic or 3D camera 40, which is configured to obtain or capture a stereoscopic or three-dimensional (3D) image
Ohtsuka, Gardner and Yang are directed to systems of obtaining and displaying stereoscopic imaging to viewers. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, by incorporating the computer architecture for multiuser communication using stereoscopic imaging as provided by Yang, using known electronic interfacing and programming techniques. The medication merely applies a known technique for a distributed, multi-computer system that uses display devices to present stereoscopic images using a display as in the base device. The modification yields predictable results of merely using a type of display device for presenting corrected/improved stereoscopic imagery, within a particular computer application, e.g. video conferencing using multiple stereoscopic display devices such as the base device. One of ordinary skill in the art would have found it obvious to merely replicate the display technique and device in place of the multiple devices. Moreover, the modification results in an improved stereoscopic display system by allowing for additional desired functionality, namely providing a practical application of video conferencing or other video-based applications between users to a system that presents improved stereoscopic images on a end-user’s display device.
The only limitation not taught by Ohtsuka modified by Gardner and Yang is that the depth conflict is based on a tracked head position of a user viewing the image content. However, determining viewports of a limited display showing stereoscopic images, and determining differing positions based on the boundaries of a changing viewpoint of a user as the user moves their head relative to a display was known at the time of the effective filing date of the claimed invention.
Lee discloses:
the depth conflict being determined based on a tracked head position of a user viewing the image content (Lee, Fig. 1 and ¶43 discloses positional arrangement of viewing window based on head and eye of user; ¶53 discusses the physical limitations of a display system, such that model outside a region cannot be shown, where region can by user-restricted by the use of a clipping box – Fig. 4;
¶71 of Lee discloses the concept of detecting depth conflicts resulting from parallax views of objects from changes in appearance of virtual objects based on motions of the user’s head and eyes, which are tracked by the system, and performing a rendering operation to eliminate the conflict (in this case transparency):
However, in a stereo environment with additional depth cues, such as, for example, parallax (i.e., the change of appearance in response to motions of the user's head and eyes, which are tracked by the system), the location of an object rendered in such translucent manner may appear very definite to the user, making this a useful technique for placing one object within another. It permits the user's visual system to construct (perceive) a consistent model of what is being seen. Thus, suppose that from a given viewpoint two opaque objects are rendered, the first of which geometrically occludes the second by having parts which block lines of sight to parts of the second, but the second is rendered after the first, opaquely replacing it in the display. The user is faced with conflicting depth cues. Stereopsis (plus parallax and perspective if available) indicate the second as more distant, while occlusion indicates that it is nearer. However, if the second object is translucently rendered, the visual system can resolve the conflict by perceiving that the second is visible through the first, as though the first were translucent to light from (and only from) the second object.
The key teaching of Lee is that it is known that a depth conflict in stereoscopic displays using parallax can be determined based on a changing viewpoint of the user, namely by tracking movement of the users head and eyes by the system and resolving resulting conflicts from the changed viewpoint; Note ¶30 of Lee disclosing use with display screen using prisms, aligned filters or holography)
Ohtsuka, Gardner, Yang, and Lee are directed to systems of obtaining and displaying stereoscopic imaging to viewers. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, and incorporating the computer architecture for multiuser communication using stereoscopic imaging as provided by Yang, by including conflict detection based on user head movement as provide by Lee using known electronic interfacing and programming techniques. The modification results in an improved stereoscopic display system by allowing a user to move naturally while still being able to view properly rendered images.
Regarding claim 1, the system of claim 12 performs substantially the same method as claim 1 and as such claim 1 is rejected based on the same rationale as claim 12.
Regarding claim 17, the system of claim 12 includes substantially the same elements as recited by claim 17 and therefore claim 17 is rejected based on the same rationale as claim 12 set forth above. The cited prior art also teaches a non-transitory computer readable medium having instructions stored thereon, the instructions, when executed by a processor, cause a computing device to carry out operations, such as in Ohtsuka [1:10-16] discussing the device and Yang, Fig. 1, ¶40 and ¶43 discussing the components of device for implementing system. Ohtsuka, Gardner and Yang are combinable for the same reasons as set forth above for claim 12.
Regarding claim 15, Ohtsuka modified by Gardner further discloses:
wherein resolving the depth conflict includes generating the at least one user interface element as a frame surrounding the image content, the at least one frame being adaptive to accommodate movements depicted in the image content. (Gardner, p. 6, Figs. 5-A and 5-B and ¶¶ 1-2: “If the object is moving, this virtual proscenium may be correspondingly moved, updated, or actively animated over a range of frames to continuously prevent the violation. This is an advantage to a fully articulated approach, which is critical for the dynamic nature of motion pictures, but was not an issue in static photography. (See Figure 6) If there is a substantial window violation on only one side of the image, the virtual proscenium may be rotated to position that side of itself in front of the violating object. (See Figure 6) This rotation of the virtual proscenium may also be animated during the shot or the scene. This rotation alters the virtual proscenium (and stereo window) to be nonparallel to the screen plane, for the first time in motion pictures.”)
Both Ohtsuka and Gardner are directed to using virtual viewing frame elements to improve stereoscopic images based on edge violations. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, by utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, using known electronic interfacing and programming techniques. The modification results in an improved display of stereoscopic images for moving images by allowing for removal of dynamic window violations, diminishing eye fatigue, and maintaining good image quality (see p. 6 of Gardner discussing the dynamic floating window solving important problems).
Regarding claim 4, the system of claim 15 performs substantially the same method as claim 4 and as such claim 4 is rejected based on the same rationale as claim 15.
Regarding claim 5, Ohtsuka modified by Gardner further discloses:
wherein a side of the frame that corresponds to the at least one portion that extended beyond the boundary is placed in a different plane parallel to, and in front of, the remainder of the frame to generate a visually perceived tilt of the frame from vertical to a non-zero angle to the vertical (Gardner, p. 6, Figs. 5-A and 5-B and ¶¶ 1-2: proscenium rotated to position that side of itself in front of the violating object, where rotation alters the virtual proscenium (and stereo window) to be nonparallel to the screen plane)
Both Ohtsuka and Gardner are directed to using virtual viewing frame elements to improve stereoscopic images based on edge violations. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, by utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, using known electronic interfacing and programming techniques. The modification results in an improved display of stereoscopic images for moving images by allowing for removal of dynamic window violations, diminishing eye fatigue, and maintaining good image quality (see p. 6 of Gardner discussing the dynamic floating window solving important problems).
Claim(s) 7, 9, 10, 16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Ohtsuka (US 5,847,870) in view of
Gardner, Brian, “The Dynamic Floating Window: a new creative tool for 3D movies”, Proceedings Volume 7863, Stereoscopic Displays and Applications XXII; 78631A (2011),
Yang (US 2017/0295357 A1) and
Lee et al. (US 2004/0233222 A1) in further view of
Kudo et al. (US 2012/0056883 A1).
Regarding claim 16, the limitations included from claim 12 are rejected based on the same rationale as claim 12 set forth above. Further regarding claim 16, Kudo discloses:
wherein the user interface element includes a blurred overlay, the blurred overlay beginning at the boundary and ending at a predefined location associated with a size of a display device depicting the image content, wherein a blur radius associated with the blurred overlay is increased at a threshold distance from the boundary. (Kudo, ¶77: display discrepancy determination determines whether image for eye exceeds screen display; ¶80: image adjustment unit 14 specifies a non-matching portion of the object image for a right eye and the object image for a left eye based on the information provided by the display discrepancy determination unit 13; ¶¶82-83 and 85 disclose adjusting transparency in non-matching portion; ¶90: modification of image adjustment unit adjusts blur instead of transparency using Gaussian filter; Fig. 12B and ¶98: amount of effect is greater based on distance from boundary 34R)
Ohtsuka, Gardner, Yang, Lee and Kudo are directed to systems of obtaining and displaying stereoscopic imaging to viewers. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, using the computer architecture for multiuser communication using stereoscopic imaging as provided by Yang, and the conflict detection based on user head movement as provide by Lee by further using the blurring technique of at edge violations as taught by Kudo, using known electronic interfacing and programming techniques. The modification results in a smoother transition that is more visually pleasing and better fitting the images into a user’s surroundings (see e.g. Kudo ¶85).
Regarding claim 7, the system of claim 16 performs substantially the same method as claim 1 and as such claim 7 is rejected based on the same rationale as claim 16.
Regarding claim 20, the system of claim 16 includes substantially the same elements as recited by claim 20 and therefore claim 20 is rejected based on the same rationale as claim 16 set forth above.
Regarding claim 9, the claim recites wherein the blurred overlay is oval- shaped, which is merely an aesthetic design choice, rather than a functional limitation. With regard to the claim, Ohtsuka modified by Gardner, Yang, Lee and Kudo teaches all the limitations except that particular shape of the blur overlay as claimed. However, it would have been obvious to one of ordinary skill in the art at the time of the invention to use different shaped blurred effects, as the applicant has not disclosed that using an oval in place of any other shape provides an advantage, solves any stated problem, or is for any particular purpose. Applicant’s specification even states that any shape can be used (see US PG-Pub, ¶120, where shapes can be square, rectangle, oval, semi-circle, semi-oval, trapezoidal, etc.). The use of the particular shape is merely to provide an aesthetic look alone, and any shape would perform would perform equally well along the edging to solve the addressed problem of edge violations in stereoscopic imagery (i.e. the focus is the display itself and fatigue or distortion to user’s ability to see the images themselves as projected, as opposed to an aesthetic design feature to the effect). Accordingly, use of an oval for the blurred overlay is deemed to be a design consideration which fails to patentably distinguish over the prior art of Ohtsuka modified by Gardner, Yang and Kudo.
Regarding claim 10, Ohtsuka modified by Gardner, Yang, Lee and Kudo further discloses:
Visual correction effect from a left central portion of the overlay to a left edge of the image content and from a right central portion of the overlay to a right edge of the image content (Ohtsuka, Figs. 1 and 4A-4B discloses the visual frame and affected regions having the sections marked off on sides of viewing plane Ps)
Gardner also discloses visual correction effect from a left central portion of the overlay to a left edge of the image content and from a right central portion of the overlay to a right edge of the image content (Garnder, pp. 7-8, Figs. 7C and 8A)
Kudo teaches:
wherein the blurred overlay is gradient blur graduating from a central portion of the blurred overlay to an edge of the image content (Kudo, Fig. 12B and ¶98: amount of effect is greater based on distance from boundary)
Examiner notes that merely performing the blur on both sides is merely an obvious rearrangement of parts. However, the combination of references, wherein a visual frame is utilized on both sides of an image as a floating window or proscenium effect to correct for the visualization of the stereoscopic image, combined with the known gradient blur effect for the edges of the images in place of other effects, teaches and renders obvious the claimed invention.
Ohtsuka, Gardner, Yang, Lee and Kudo are directed to systems of obtaining and displaying stereoscopic imaging to viewers. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention and with a reasonable expectation of success, to modify the system and method for using a floating frame for removing window violations of objects in stereoscopic images as provided by Ohtsuka, utilizing a floating frame that adjusts based on the movement of virtual objects displayed in stereoscopic images as provided by Gardner, using the computer architecture for multiuser communication using stereoscopic imaging as provided by Yang, and the conflict detection based on user head movement as provide by Lee by further using the blurring technique of at edge violations as taught by Kudo, using known electronic interfacing and programming techniques. The modification results in a smoother transition that is more visually pleasing and better fitting the images into a user’s surroundings (see e.g. Kudo ¶85).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM A BEUTEL whose telephone number is (571)272-3132. The examiner can normally be reached Monday-Friday 9:00 AM - 5:00 PM (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DANIEL HAJNIK can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM A BEUTEL/Primary Examiner, Art Unit 2616